Reproduced  From 


rm 


UNITED  STATES  AIR  FORCE  TECHNICAL 
ORDER  ACQUISITION*  WHAT  ARE  THE 
PROBLEMS  AND  HOW  CAN  THEY 
BE  CORRECTED? 

THESIS 

Thomas  D.  Brovn,  Jr.  Dennis  R.  Lyon 

Captain,  USAF  Captain,  USAF 

AF IT/GLM/LSM/84S-6 


<0. DCCOQOZMS 


ELECTEI 
NOVI  884 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  UNIVERSITY  ^ 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 


Wright-Patterson  Air  Force  Base,  Ohio 


DISTRIBUTION  STATEMENT  A 

Approved  fat  pub!  a  release) 
Distribution  Unlimited 


84  10  30  003 


AFIT/GLK/LSM/84S-6 


UNITED  STATES  AIR  FORCE  TECHNICAL 
ORDER  ACQUISITIONS  WHAT  ARE  THE 
PROBLEMS  AND  HOW  CAN  THEY 
BE  CORRECTED? 


THESIS 


Thomas  D.  Brown#  Jr.  Dennis  R.  Lyon 

Captain#  USAF  Captain#  USAF 


AF IT/GLM/LSM/84S-6 


DTIC 


ELECTE 
NOVI  1984 


A 


B 


Approved  for  public  release;  distribution  unlimited 


The  contents  of  the  document  are  technically  accurate,  and  no 
sensitive  items,  detrimental  ideas,  or  deleterious  informa¬ 
tion  are  contained  therein.  Furthermore,  the  views  expressed 
in  the  document  are  those  of  the  author  (s)  and  do  not  "fe®Srily 
reflect  the  views  of  the  School  of  Systems  and  Logistics,  the 
”r  “iveMitj!  ?he  united  States  Air  Force,  or  the  Department 

of  Defense. 


AF IT/GLM/LSM/84S-6 


UNITED  STATES  AIR  FORCE  TECHNICAL  ORDER  ACQUISITION* 
WHAT  ARE  THE  PROBLEMS  AND  HOW 
CAN  THEY  BE  CORRECTED? 


THESIS 


Presented  to  the  Faculty  of  the  School  of  Systems  and  Logistics 
of  the  Air  Force  Institute  of  Technology 
Air  university 

In  Partial  Fulfillment  of  the 
Requirements,  for  the  Degree  of 
Master  of  Science  in  Logistics  Management 


Thomas  D.  Brown#  Jr.#  B.B.A. 
Captain#  USAF 


Dennis  R.  Lyon#  B.A. 
Captain#  USAF 


September  1984 


Approved  for  public  release;  distribution  unlimited 


Preface 


Our  interest  in  the  subject  of  Technical  Orders  (TOs) 
stems  from  our  experience  as  operators  and  maintainers.  We 
have  both  been  frustrated  by  TGs  that  did  not  seem  to  be  as 
good  as  they  could  have  been,  and  the  literature  indicates 
that  problems  in  the  acquisition  process  are  at  the  root  of 
the  problems.  It  was  with  an  eye  toward  improving  the  end 
product  that  we  undertook  this  research,  in  the  hope  that 
our  efforts  would  benefit  future  TO  users. 

We  were  encouraged  during  our  research  by  the 
enthusiasm  of  our  advisor,  Mr.  Art  Munguia.  His  interest 
was  more  than  academic,  and  his  knowledge  of  the  TO 
acquisition  process  was  a  great  asset  to  our  research.  We 
also  benefited  from  the  experience  and  judgement  of  our 
readers,  Maj  John  Stibravy  and  Dr.  Robert  Weaver. 

As  with  any  project  of  this  magnitude,  we  owe  a  great 
debt  to  cur  families  for  their  patience  and  understanding 
during  those  times  when  we  were  laboring  with  this  research. 
We  sincerely  hope  our  efforts  are  worthy  of  their 
sacrifice. 

Thomas  D.  Brown,  Jr.  Dennis  R.  Lyon 
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Abstract 

The  objective  of  this  research  was  to  identify  problems 
with  the  acquisition  of  U.S.  Air  Force  technical  orders  (TOs) 
and  to  identify  changes  to  the  TO  acquisition  process  that 


could  solve  those  problems. 

A  telephone  survey  of  A.F.  policy  makers #  program  man¬ 
agers#  integrated  logistics  support  managers#  and  technical 
order  acquisition  managers  was  accomplished.  The  low  experi¬ 
ence  level  of  personnel  assigned  to  TO  acquisition  jobs#  and 
coordination  and  communication,  problems  were  found  to  be  the 
most  significant  problems  of  the  technical  order  acquisition 
process.  Inadequate  manning  and  the  need  for  earlier  plan¬ 
ning  for  TO  acquisition  were  also  found  to  be  problems. 

Five  solutions  to  these  problems  were  found  to  be  valid 
and  were  recommended  for  implementation.  They  were  the  es¬ 
tablishment  of  a  centralized  TO  management  agency#  the  es¬ 
tablishment  of  TOs  as  a  separate  product#  the  development  of 
"skeleton"  documents#  the  development  of  a  handbook  out¬ 
lining  responsibilities,  and  the  establishment  of  a  TO  ac¬ 
quisition  management  career  field.  The  establishment  of  a 
centralized  TO  management  agency  was  found  to  be  the  most 
needed  solution.  An  implementation  plan  was  presented  for 


all  recommended  solutions. 
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UNITED  STATES  AIR  FORCE  TECHNICAL  ORDER  ACQUISITION: 

WHAT  ARE  THE  PROBLEMS  AND  HOW  CAN  THEY  BE  CORRECTED? 

I.  Introduction 

Gene ra 1  Issue 

Technical  Orders  (TOs)  are  the  key  to  the  transfer  of 
essential  technical  information  from  design  engineers  to 
operators  and  technicians.  It  is  imperative  that  system 
Program  Managers  (PMs)  initiate  early  planning  and  coor¬ 
dination  for  the  acquisition  of  accurate#  complete#  and 
understandable  TOs  for  all  new  weapon  systems.  A  literature 
review  show;  that  inadequate  TOs  have  been  acquired  for 
several  Air  Force  weapon  systems.  Inadequacies  were  found 
with  the  C-141,  the  B-52  G/H#  and  Inter-Continental  Bal¬ 
listic  Missile  (ICBM)  maintenance  or  operator  technical 
manuals.  These  inadequate  TOs  have  resulted  in  increased 
program  cost#  reduced  operational  readiness#  and  increased 
operations  and  maintenance  costs.  In  fact,  from  1979  through 
1933,  investigations  have  identified  logistics  factors  t  be 
indirectly  involved  in  35  in-flight  mishaps.  Included  *s 
logistics  factors  are  incorrect,  incomplete,  or  inadequate 
operations  and  maintenance  manuals  or  procedures.  It  must  be 
emphasised  that  logistics  factors  were  indirectly  involved 
and  were  not  listed  as  the  primary  cause  of  any  of  the  35 
mishaps.  The  total  repair  costs  for  the  35  mishaps  (1)  was 
$128,965,309.  Interviews  with  technical  order  acquisition 


experts  have  Identified  several  problems  vithin  the  tech¬ 
nical  order  acquisition  process  that  could  result  in  even 
more  inadequate  TOs  being  acquired  in  the  future.  These 
technical  order  acquisition  problems  must  be  solved.  The 
literature  review  and  additional  interviews  with  technical 
order  acquisition  experts  show  that  recommendations  to 
change  the  technical  order  acquisition  process  have  been 
suggested  by  virtually  every  management  level  within  the  Air 
Force.  Knowledge  of  the  problems  and  the  recommended  solu¬ 
tions  to  those  problems #  as  noted  in  this  thesis#  will  as¬ 
sist  the  Air  Staff  and  other  managers  who  oversee  the  tech¬ 
nical  order  acquisition  process  to  make  improvements  in  the 
future. 

General  Problem 

Changes  need  to  be  made  to  the  technical  order  acqui¬ 
sition  process  that  will  reduce  the  general  systems  manage¬ 
ment  problems  currently  identified  with  that  acquisition 
process  and  reduce  the  number  of  inadequate  technical  orders 
being  acquired. 

Background 

Two  questions  must  be  answered  before  designing  a  re¬ 
search  project  aimed  at  identifying  recommended  changes  to 
the  technical  order  acquisition  process  that  will  sdlve 
technical  order  acquisition  problems.  The  questions  are 
aimed  at  identifying  exactly  which  problems  can  be  solved  by 


changes  to  the  technical  order  acquisition  process.  The  two 
questions  are: 

1.  What  technical  order  inadequacies  are  causing  the 
user  problems? 

2.  What  are  the  problems  identified  by  technical 
order  acquisition  experts? 

1 .  Technical  Order  Inadequacies .  A  literature  review 
indicates  that  research  into  user  oriented  technical  order 
inadequacies  has  been  conducted  by  the  Air  Force  Human  Re¬ 
sources  Laboratory  (2)#  Applied  Science  Associates#  Inc. 

(3)#  and  Air  Command  and  Staff  College  students  (4).  Inad¬ 
equacies  have  been  found  in  seven  technical  order  character¬ 
istics  by  one  or  more  of  the  research  projects.  Those  seven 
characteristics  are: 

1.  Completeness 
.  2 .  Accuracy 

3 .  Relevance 

4.  Retrievability 

5.  Understandability 

6.  Compliance 

7 .  Economy 

When  any  of  these  characteristics  are  missing  from 
technical  orders,  inadequate  technical  orders  can  result. 

The  user  can  have  problems  performing  technical  order 
directed  tasks  when  using  inadequate  technical  orders.  Tech¬ 
nical  orders  are  purchased  through  the  technical  order 
acquisition  process.  Inadequate  technical  orders  are  a 


result  of  inadequacies  within  the  technical  order  acqui¬ 
sition  process. 

2.  Technical  Order  Acquisition  Problems .  Technical 
order  acquisition  problems  have  been  identified  through 
research  sponsored  by  the  Deputy  Secretary  of  Defense  (5), 
and  the  Air  Force  Logistics  Management  Center  (6).  As  part 
of  this  research^  members  of  the  U.  S.  Air  Force  Centralized 
Technical  Order  Management  Group  Executive  Committee  (CTOM) 
were  interviewed.  The  CTOM  consists  of  the  top  technical 
order  specialists  from  each  of  the  major  commands  and  is 
chaired  by  the  top  technical  order  specialist  cm  the  Air 
Staff.  Other  interviews  were  conducted  with  technical  order 
experts  from  the  Defense  Material  Specifications  and  Stan¬ 
dards  Office  (7)  and  from  the  Air  Force  Systems  Command 
Systems  Program  Offices  (SPOs).  The  results  of  the  research 
and  interviews  indicate  that  the  experts  believe  that  there 
are  four  major  problem  areas  within  the  current  technical 
order  acquisition  system.  Those  four  problem  areas  are: 

_ 1.  Technical  order  planning  does  not  begin  early  enough 

in  the  acquisition  cycle. 

2.  Coordination  and  communication  between  all  agencies 
involved  in  the  weapon  system  acquisition  process  is 
not  adequate. 

3.  Manpower  dedicated  to  technical  order  acquisition 
is  not  adequate. 

4.  Training  and  assistance  for  technical  order 
acquisition  personnel  is  not  adequate. 

The  technical  order  experts  have  proposed  recommen¬ 
dations  aimed  at  solving  these  problems. 
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Scope  of  Research 

For  the  purposes  of  this  research,  the  term  "technical 
order"  will  refer  only  to  the  technical  manuals  used  to 
maintain  or  operate  Air  Force  equipment.  The  problems  and 
systems  studied  in  this  research  project  will  be  only  those 
that  have  to  do  with  maintenance  or  operations  technical 
orders. 

The  major  focus  of  this  research  is  towards  major  Air 
Force  weapon  systems  since  the  acquisition  process  for  them 
is  more  complex  than  jthat  for  non-major  systems;  however, 
the  problems  being  studied  are  not  unique  to  major  systems. 
The  changes  to  the  technical  order  acquisition  process  that 
will  be  recommended  ajs  a  result  of  this  research  will  be 
applicable  to  major  and  non-major  weapon  systems  as  well  as 
to  support  equipment.! 

i 

This  research  project  will  not  attempt  to  examine  every 
individual  technical  order  related  problem.  The  problems  are 
too  numerous.  The  individual  problems  have  been  examined, 
and  four  basic  deficiencies  have  been  found  to  be  respon¬ 
sible  for  the  majority  of  those  problems.  For  example  this 
research  will  not  be  directly  targeted  at  problems  with 
validation  or  selection  of  format;  however,  the  four  areas 
that  will  be  examined  have  been  found  to  contribute  to  many 
of  the  validation  and  format  selection  problems.  The  four 
basic  deficiencies  that  will  be  examined  in  this  research 
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project  are  the  research  questions  that  form  the  research 
objectives  of  this  project. 


The  final  limitation  of  this  research  project  is  the 
nature  of  the  project.  This  research  is  exploratory  in 
nature.  As  an  exploratory  project#  no  hypothesis  will  be 
tested.  The  intent  of  this  research  is  to  identify  the 
nature  of  the  problems  with  the  technical  order  acquisition 
process  and  to  identify  potential  changes  to  the  acquisition 
system  that  can  solve  those  problems.  The  changes  that  will 
be  recommended  as  a  result  of  this  research  will  be  hypoth¬ 
eses  for  future  research. 

Research  Ob  jectives 

The  objective  of  this  research  is  to  identify  problems 
with  the  acquisition  of  U.  S.  Air  Force  technical  orders# 
and  identify  possible  changes  to  the  technical  order  acqui¬ 
sition  process  that  will  solve  those  problems.  This  objec¬ 
tive  will  be  accomplished  by  examining  four  research 
questions. 

Research  Questions  ' 

The  four  research  questions  ares 

1.  How  can  technical  order  acquisitions  be  integrated 
into  earlier  phases  of  the  weapon  system  acquisition 
cycle? 

2.  How  can  communication  and  coordination  between 
technical  order  acquisition  agencies  and  other 
acquisition  agencies  be  improved? 

3.  How  can  manning  and  retention  of  technical  order 
acquisition  personnel  be  improved? 
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9. 


4.  Hov  can  training  and  knowledge  of  technical  order 
acquisition  personnel  be  improved? 

Answering  these  four  research  questions  should  result 
in  recommended  changes  to  the  technical  order  acquisition 
system.  Those  recommended  changes  will  in  effect  be  hypoth¬ 
eses  that/  once  implemented  and  tested/  will  either  solve 
the  identified  problems  or  lead  to  further  hypotheses  to 
solve  the  problems. 

The  following  chapter  will  present  an  investigation  of 
technical  order  related  literature.  That  investigation  will 
be  used  as  a  basis  for  this  research  project. 


7 


II.  Literature  Review 


This  literature  review  will  be  concerned  with  two  major 
areas.  The  first  area  to  be  examined  will  be  the  research 
aimed  at  the  identification  of  technical  order  related  pro 
lems.  The  second  area  to  be  examined  will  be  the  research 
aimed  at  recommending  changes  to  the  technical  order  acqui¬ 
sition  process  to  solve  the  identified  problems.  In  addi¬ 
tion  ,  the  relationship  between  the  problems,  the  recommended 
changes  to  the  technical  order  acquisition  process,  and  the 
objectives  of  this  research  project  will  be  discussed. 

The  Problems 

The  literature  concerned  with  technical  order  related 
problems  can  be  divided  into  two  major  categories.  Those 
categories  are  the  literature  aimed  at  user-oriented  prob¬ 
lems  caused  by  technical  orders,  and  the  literature  aimed  at 
the  problems  with  the  technical  order  acquisition  process. 

User-Oriented  Problems .  Ultimately,  problems  with 
maintenance  TOs  become  user-oriented  problems.  If  mainte¬ 
nance  personnel  use  inaccurate,  incomplete,  or  incomprehen¬ 
sible  TOs,  job  performance  will  suffer. 

Hatteric  and  Price  (2)  found  that  user-oriented  prob¬ 
lems  result  when  inadequate  TOs  are  used  to  perform  mainte¬ 
nance  tasks.  These  user-oriented  problems  can  surface  in 
several  different  ways  (2:137).  1)  Maintenance  personnel  can 


8 


use  the  Inadequate  TOs  to  accomplish  the  task  vhich  could 
result  in  inadequate  maintenance  performance.  2)  Mainte¬ 
nance  personnel  can  disregard  the  TOs  and  use  their  own 
resources  to  perform  the  task  which  can  result  in  inconsist 
ent  maintenance  performance  that  will  vary  with  each  techni 
cian.  3)  Maintenance  personnel  can  rely  on  supervisory 
personnel  to  develop  local  policy  to  accomplish  the  task 
which  can  result  in  satisfactory  performance#  but  only  for 
the  area  of  responsibility  of  the  supervisor  involved.  No 
matter  which  approach  is  taken#  inadequate  technical  orders 
cause  problems  to  the  user. 

Andrew  Chenzoff  (3)  identified  seven  characteristics 
that  an  adequate  technical  order  must  posses.  Chenzoffa* 
seven  characteristics  are: 

1.  Completeness-- How  to  make  sure  that  the 
performance  aids  [TOs]  contain  all  of  the 
information  that  the  user  needs  to  know. 


2.  Accuracy — How  to  make  sure  that  the  information 
the  user  obtains  from  the  performance  aids  [TOs] 
is  factually  accurate. 

3.  Relevance— How  to  make  sure  the  performance  aids 
[TOs]  contain  as  little  information  as  possible 
that  has  no  value  to  any  user#  while  (Insuring  all 
users'  needs  are  met. 


4.  Ret rievabi li ty— How  to  make  sure  that 'the  user 
can  quickly  find  and  extract  the  total  body  of 
information  he  needs  for  task  performance. 

5.  Understandability — How  to  make  sure  that  the  user 
can  correctly  understand  and  interpret  the 
information  he  finds. 

6.  Compliance — How  to  make  sure  that  the  user  reads 
and  uses  the  information  and  instructions 
provided . 


I 
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7.  Economy — How  to  provide  effective  performance 
aids  ( Instruments  that  cope  with  the  above-cited 
problems)  within  reasonable  limits  of  cost  [3:13]. 

When  any  of  these  characteristics  are  missing  from  a 

technical  order*  problems  can  occur. 

Completeness.  Many  problems  can  occur  if  required 

information  is  omitted  from  TOs.  Two  examples  ares 

Reports  from  the  Flight  Test  Center  at  Edwards  AFB 
indicate  maintenance  and  operating  manuals  received 
with  equipment  for  test  and  evaluation  in  many  cases  do 
not  provide  adequate  instructions  to  accomplish 
required  tasks  [8:6]. 

When  used  by  Missile  Maintenance  personnel*  SAC  Civil 
Engineering  Manuals  (CEMs)  cause  additional  maintenance 
dispatches*  wasted  manhours*  and  unnecessary  removal 
and  replacement  of  serviceable  components  due  to  a  lack 
of  adequate  instructions  [8:1]. 

Accuracy.  Many  problems  can  occur  if  required 

|  information  is  not  accurate.  For  instances 

* 

In  the  B-52  modification  Inspect  and  Repair  As 
Necessary  (IRAN)  program  at  Kelly  Air  Force  Base  ... 
it  was  discovered  that  the  technical  manuals  were  not 
adequate  [accurate]  in  checking  out  the  avionics 
subsystems.  Aircraft  repair  and  return  schedules  had 
tc  be  slipped  since  maintenance  could  not  be  completed 
[9:5]. 

The  B-52  navigational  test  equipment  was  aligned  at 
Kelly  AFB,  Texas,  wi  Ji  TO  specified  coordinates  for  Wichita* 
Kansas  (home  of  the  contractor). 

Relevance.  TOs  must  contain  sufficient  infor¬ 
mation  to  allow  the  maintenance  personnel  to  successfully 
complete  the  task.  A  survey  of  248  flightline  and  shop  tech¬ 
nicians  assigned  to  maintain  C-141  aircraft  at  Charleston 
Air  Force  Base*  South  Carolina,  and  Norton  Air  Force  Base, 
California  shows: 
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Technicians  expressed  a  strong  need  for  good  TOs 
that  would  serve  both  as  a  training  and  on-the-job 
performance  aids.  Based  on  the  opinions  expressed  in 
this  and  the  Losse  et  al.  (1962)  survey,  the 
observation  can  be  made  that  conventional  TOs  do  not 
fulfill  adequately  the  needs  of  maintenance  technicians 
and  personnel. 

Opinions  of  maintenance  technicians  and  personnel 
suggested  that  the  primary  need  is  for  step-by-step 
information  detailing  how  to  do  a  particular  job.  In 
essence,  the  stress  should  be  less  on  theoretical  and 
more  on  pragmatic  job-related  information.  In  addition, 
there  was  an  expressed  desire  for  more  detailed  il¬ 
lustrations  [10s 6-7]. 

Retrievability.  The  problem  of  having  to  use 

several  different  manuals  for  one  Missile  Maintenance  task 

(8:4)  is  an  example  of  a  retrievability  problem: 

Many  maintenance  tasks  on  the  Minuteman  Missile  require 
the  technician  to  use  several  different  manuals  to  be 
in  compliance  with  Air  Force  procedures  [8:4]. 

Even  if  the  required  information  is  available,  the  job 

% 

won't  get  done  if  the  maintenance  personnel  can't  find  it. 

Understandability.  A  C-141  experimental  TO  proj¬ 
ect  shows  how  understandability  of  a  given  TO  format  is 
related  to  the  skill  level  of  the  user  and  that  performance 
drops  when  the  TO  is  not  understood  by  the  user. 

The  study  (2:119)  compared  the  number  of  maintenance 
troubles  identified  and  the  number  of  unnecessary  spare 
parts  replaced  by  personnel  using  three  different  kinds  of 
TOs.  Two  of  the  TOs  used  in  the  study  were  experimental, 
easier-to-understand  formats,  while  the  other  was  the 
conventional  TO  already  used  for  the  maintenance  tasks  being 
performed.  The  maintenance  personnel  performing  the  tasks 
had  a  high  electronics  aptitude  (80th  percentile  and  above). 
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The  maintenance  personnel  using  the  two  experimental , 
easier-to-under stand  TOs  performed  considerably  better  than 
those  using  conventional  TOs  (2:123-124). 

Understandable  TOs  can  improve  maintenance  performance. 

Compliance.  Technical  orders  must  be  complete, 
accurate,  relevant,  retrievable,  and  understandable.  If  not, 
the  technical  order  will  not  be  appropriate  for  the  desired 
task.  Chenzoff  (3),  found  that  if  the  user  can  not  accom¬ 
plish  the  task  using  a  technical  order,  he.  will  not  use  one. 
Compliance  with  technical  orders  should  improve  if  the  tech¬ 
nical  order  is  complete,  accurate,  relevant,  retrievable, 
and  understandable  (3).  However,  technical  order  adequacy  is 
not  the  only  factor  affecting  compliance.  "Don't  forget  the 
natural  tendency  to  skip  the  TO  when  in  a  hurry.  Using  the 
TO  takes  discipline,  and  more  discipline  -  ask  any  mainte¬ 
nance  officer  [11]!"  Acquiring  adequate  technical  orders 
will  not  guarantee  compliance;  however,  inadequate  technical 
orders  can  cause  non-compliance. 

Economy .  The  Air  Force  dc  j  not  have  an  unlimited 
budget.  Technical  orders  must  be  made  as  adequate  as  pos¬ 
sible  within  the  budgets  allowed  for  them.  The  number  of 
TOs  needed  and  the  level  of  detail  required  in  the  format 
both  contribute  to  the  cost  of  a  TO.  Generally,  a  decision 
must  be  made  as  to  the  number  and  the  detail  required  to 
give  maintenance  personnel  the  information  they  require  to 
successfully  perform  their  duties.  When  the  money  is  not 
available  to  meet  the  projected  TO  needs,  compromise 
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usually  results.  This  compromise  can  result  in  problems  of 
completeness#  accuracy#  relevance#  retrievability#  and 
under standabi li ty . 

Remarks.  Technical  orders  can  cause  problems  for 
the  user  if  they  are  not  complete#  accurate#  relevant# 
retrievable#  and  understandable,  .tot  only  must  technical 
orders  contain  all  of  the  preceding  characteristics#  they 
must  also  be  affordable.  Technic*. I  orders  are  procured 
through  a  system  that  will  be  referred  to  as  the  technical 
order  acquisition  process.  This  technical  order  acquisition 
process  is  responsible  for  supplying  the  users  with  appro¬ 
priate  technical  orders  for  their  assigned  tasks.  Deficien¬ 
cies  within  the  technical  order  acquisition  process  can 
result  in  the  acquisition  of  inadequate  technical  orders. 
Technical  order  acquisition  experts  are  concerned  that 
several  deficiencies  within  the  technical  order  acquisition 
process  will  lead  to  inadequate  technical  orders  being 
acquired  in  the  future. 

Technical  Order  Acquisition  Problems.  Hatterick  and 
Price  (2)  found  that  data  collected  over  the  last  20  years 
suggests  that  the  performance  of  maintenance  personnel  can 
be  improved  through  the  acquisition  of  adequate  technical 
orders  (TOs)  (2:133).  Kirsch  (4)  supported  this  finding  when 
he  wrote: 

It  is  evident  that  a  great  deal  of  research  has  been 
devoted  to  devising  methods  to  improve  job  performance 
aids  (TOs).  It  is  also  evident  that  the  evolution  of 
acquisition  policy  is  being  influenced  by  research 
findings;  however,  it  takes  a  great  deal  of  time  for 

13 

7 


; 


/ 


revised  policy  to  filter  down  through  the  various 

policy  levels  [4:33]. 

Literature  on  technical  order  acquisition  process 
problems  is  virtually  non-existent.  A  major  objective  of 
this  research  project  is  to  identify  those  problems.  Top 
technical  order  experts  throughout  the  Air  Force  were  inter¬ 
viewed  either  in  person  or  by  telephone.  Those  interviews# 
as  well  as  a  review  of  Regulations*  Memorandums#  and  Direc¬ 
tives#  revealed  four  major  problems  with  the  technical  order 
acquisition  process  that  concern  the  experts.  Those  four 
problems  are: 

1.  Technical  order  planning  does  not  begin  early 
enough  in  the  acquisition  cycle  of  Air  Force 
systems . 

2.  Coordination  and  communication  between  all 
agencies  involved  in  the  weapon  system 
acquisition  process  is  not  adequate. 

3.  Manpower  dedicated  to  technical  order  acquisition 
is  not  adequate. 

4.  Training  and  assistance  for  technical  order 
acquisition  personnel  is  not  adequate. 

The  experts  indicated  that  these  four  problems  are  the 
major  problems  with  the  system  today.  They  also  felt  that 
these  problems  are  responsible  for  the  majority  of  the 
user-oriented  technical  order  inadequacies  of  the  past  (12). 

Need  for  Sarliier  Planning.  Increased  interest  in 
supportability  and  reliability  as  part  of  life  cycle  cost 
considerations  of  systert  acquisition  programs  has  resulted 
in  the  need  for  earlier  planning  of  TO  acquisition  objec¬ 
tives  . 
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Initiative  No.  9:  System  Support  and  Readiness  of  t.ie 

Deputy  Secretary  of  Defense  Memorandum  of  the  Acquisition 

Improvement  Program  .( AIP )  Second  Year  End  Report  states: 

Applicable  Directives  and  Instructions  (5000.1,  5000.2) 
have  been  revised  and  5000.39  is  in  revision  to 
increase  the  priority  of  support  and  readiness  in 
acquisition  programs  [13:12]. 

Recommendation  9  of  the  Deputy  Secretary  of  Defense 
Carlucci  initiatives  includes  a  major  emphasis  on  support 
issues.  Improvement  of  readiness  should  be  a  major  objective 
of  the  Administration,  and  implementation  must  take  place. 
Improvement  will  require  additional  technical  effort  and 
resources  early  in  acquisition  programs  (5:p. 9-155). 

The  technical  order  acquisition  process  must  be  plan¬ 
ned,  progressively  monitored,  and  updated  to  insure  timely 
completion  and  delivery  for  adequate  logistics  support 
(14:p.29-l).  Items  that  must  b*3  included  in  the  process  have 
been  identified  through  various  research  and  inspection 
projects.  Those  items  include: 

Any  change  to  AFR  8-2,  TO  00-5-1,  etc.  must  consider 
the  total  TO  production  effort,  not  just  the  final 
verification  exercise.  If  we  do  not  provide  the  con¬ 
tractor  with  proper  guidance  as  to  user  needs  during  TO 
development,  we  cannot  expect  to  be  completely  satis¬ 
fied  when  actual  verification  takes  place  [15]. 

Also,  earlier  attention  should  be  given  to: 

The  method  of  projecting  critical  maintenance  manpower 
skill  limitations  and  translating  these  into  design 
constraints  and  objectives  for  inclusion  in  RFPs  and 
specifications  [5 :p. 9-164], 

The  U.  S.  Air  Force  Technical  Order  Management  Group 
r:"  cutive  Committee  added: 
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Target  military  populations,  in  terms  of  skill  level, 
for  each  TO  must  be  determined  by  the  using  command 
early  in  the  acquisition  phase  and  made  part  of  the 
validation/verification  plan  by  the  TOMA.  Using 
commands  must  provide  proper  skill  ratios  during  actual 
verification  [16*5]. 

Early  and  detailed  planning  is  required  to  prepare 
the  Technical  Order  Management  Plan.  Air  Force  Logistics 
Command  /.Air  Force  Systems  Command  Pamphlet  800-34, 


Upon  setting  up  the  TO  requirements,  the  TOMA  ... 
should  prepare  a  draft  TMP  (Technical  Order  Management 
Plan).  This  draft  should  be  submitted  with  the  DID 
requiring  contractor  development  of  the  TMP  as  part  of 
the  RFP.  The  draft  serves  as  a  guide  for  the 
contractor (s)  to  follow  in  developing  the  details  to  be 
included  in  the  TMP  [ 14  *  29-2 ] . 

Increased  interest  in  supportability  and  reliability  as 
part  of  life  cycle  cost  considerations  of  system  acquisition 
programs  has  resulted  in  specific  regulations  and  directives 
for  earlier  planning  of  technical  order  acquisition  objec¬ 
tives. 

Interviews  were  conducted  with  members  of  the  U.S.  Air 
Force  Technical  Order  Management  Group  Executive  Committee 
(CTOM).  Members  of  the  CTOM  are  the  top  technical  order 
managers  from  each  Major  Command  in  the  Air  Force.  They  do 
not  believe  that  technical  order  personnel  are  being  as¬ 
signed  to  system  acquisition  programs  early  enough  in  the 
system  acquisition  cycle  to  accomplish  the  goals  required  of 
them  by  regulation  (17).  In  fact  many  acquisition  programs 
(less  than  major)  do  not  even  have  technical  order  acqui¬ 
sition  specialists  assigned  (17). 
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Implementing  regulations  is  not  easy.  The  Deputy 
Secretary  of  Defense  Memorandum  for  Improving  the  Acqui¬ 
sition  Process  (5)  states: 

A  difficulty  with  implementing  recommendations 
regarding  the  acquisition  process  is  the  great  number 
of  players  involved  to  make  implementation  succeed. 

This  requires  persistent,  intensive,  follov-up  effort 
to  make  sure  that  the  recommendations  really  do  take 
hold.  The  most  common  reason  for  non-implementation  is 
simply  that  relentless  action  on  the  part  of  top 
management  is  not  taken  to  insure  that  recommendations 
are,  indeed,  implemented.  OSD  has,  in  the  past, 
focussed  a  great  amount  of  management  attention  on 
policy  development  and  resolution.  However,  OSD  has  not 
monitored  implementation  of  the  policies  on  a  program 
basis  [5 :p. 9-171]. 

A  change  in  policy  and  acquisition  objectives  takes 

time  and  work  through  all  levels  of  management.  Getting 

support  and  TO  considerations  higher  priority  is  a  needed 

change  that  has  not  happened  yet:  * 

There  is  a  widespread  belief  that  performance  and 
schedule  are  DOD's  principal  objectives.  There  is  a 
need  for  industry  to  apply  more  of  their  design  talents 
to  reducing  reliability  and  support  problems.  Beyond 
this  is  a  need  to  improve  the  identification  and 
specification  of  maintenance  manpower  constraints  and 
for  industry  to  include  these  constraints  in  the 
designs  [5:p.9-64], 

Another  Deputy  Secretary  of  Defense  Memorandum  further 
stated : 

Cost  and  schedule  pressures  continue  to  dominate  the 
structure  and  acquisition  strategy  of  most  programs. 

Too  little  management  attention  and  resources  are 
devoted  to  structuring  the  early  design  and  test 
sequences  for  achieving  R&M  and  readiness  objectives 
and  implementing  the  most  efficient  support  strategy 
[13:12]. 


This  problem  has  also  been  identified  in  the  Technical 

Order  Lessons  Learned  Bulletin  { 8 ) ,  published  by  the  Air 

Force  Acquisition  Logistics  Center* 

The  "bottom  line"  is  that  the  magnitude  of  the  program 
and  the  management  expertise  required  to  acquire  TOs 
successfully  is  frequently  not  recognized  until  it  is 
too  late.  Early  planning  and  follow-up  are  essential  to 
a  successful  TO  program;  as  documented  in  many  of  the 
Lessons  Learned  [8*101. 

TO  Managers  are  not  always  assigned  early  enough  to 
accomplish  all  the  TO  acquisition  tasks  assigned  to  the 
TOMA.  The  TO  Manager  for  the  F-15  program  was  assigned  two 
days  prior  to  final  source  selection  review  (18).  The  T-46A 
system  program  director  stated  that  TO  considerations  for 
his  program  were  basically  the  result  of  efforts  by  himself 
and  the  DPML  prior  to  the  assignment  of  the  TO  Manager  (19). 

Technical  order  planning  does  not  begin  early  enough  in 
the  system  acquisition  cycle. 

Better  Coordination  and  Communication.  The  tech¬ 
nical  order  acquisition  system  is  not  self-contained.  Coor¬ 
dination  and  communication  between  technical  order  personnel 
and  other  system  acquisition  agencies  is  imperative* 

.  .  .  considerations  and  tradeoffs  should  be  made/  in 
coordination  with  the  appropriate  AFLC/ALC ,  by  the 
Systems  Program  Office  (SPO)  Data  (TO)  Manager/  or 
similar  office  [20*23]. 

The  best  plan  in  the  world  is  useless  if  the  con¬ 
tractor  does  not  know  what  is  wanted* 

The  contractor  should  develop  a  validation  plan  that 
states  how  and  when  TOs  will  be  validated.  The  TO 
Manager  must  review  this  validation  plan  and  ensure 
that  the  contractor  does  a  thorough  validation.  The 
validation  plan  -nust  become  part  of  the  Technical  Order 
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Publications  Plan  [TOPP  -  the  same  as  a  Technical  Order 
Management  Plan]  which  should  be  a  binding  contractual 
document  [8:6]. 

An  interview  with  Mr  John  winters  from  the  Defense 
Material  Specifications  and  Standards  Office  (7),  identified 
a  problem  with  contracts  and  MIL  Standards .  MIL  Standards 
are  the  basis  of  our  contract  system.  TO  acquisition  MIL 
Standards  are  very  complex.  One  MIL  Standard  is  like  a  tree 
with  many  branches.  One  MIL  Standard  may  refer  to  many  other 
MIL  Standards.  MIL  Standards  are  dynamic  and  are  continually 
being  revised  and  changed.  Intensive  research  and  training 
is  required  to  know  the  status  of  the  MIL  Standards  being 
used  to  acquire  TOs.  With  the  manpower  and  training  limita¬ 
tions  put  on  the  TOMA,  the  desired  level  of  MIL  Standard 
knowledge  is  not  always  achieved.  Often  resulting  in  con¬ 
tracts  with  one-time  special  case  clauses  or  contracts . that 
do  not  acquire  the  expected  product.  This  results  in  renego¬ 
tiating,  and  higher  costs.  Better  coordination  and  transfer 
of  information  between  Technical  Order  Managers  and  MIL 
Standard  specialists  is  needed  (7). 

Another  example  of  needed  coordination  is  in  the 

establishment  of  system  timetables: 

When  an  acquisition  urgency  exists,  the  impact  on 
technical  manual  preparation  should  be  determined, 
relative  to  sustaining  O&M  and  other  support  areas  and 
such  facts  presented  to  higher  echelons  to  establish  an 
adjusted  realistic  schedule.  Provide  for  utilization  of 
corrected  preliminary  manuals  until  final  manuals  are 
available  (8:19). 

If  TO  support  timetables  are  not  adjusted  to  coincide 
with  the  rest  of  the  system,  problems  similar  to  one  briefed 


to  the  Centralized  Technical  Order  Management  Group  Execu¬ 
tive  Committee  could  result:  TOs  with  critical  ballistics 

information  were  not  available  when  the  bombs  were  deliv- 

\ 

ered.  The  Armament  Division  was  responsible  for  testing  the 
weapons  and  developing  the  ballistics  data.  Due  to  manning 
cutbacks#  the  data  was  not  available  to  the  TO  contractor  in 
enough  time  to  complete  the  TO  prior  to  the  completion  of 
production  on  the  bombs  (17).  This  is  an  example  of  how 
early  planning  and  coordination  are  both  critical  to  the  TO 
acquisition  process. 

Air  Force  Systems  Command  Pamphlet  800-3,  Acquisition 
Management  ( 21 ) ,  stresses  coordination  in  the  acquisition 
process: 

Accomplishing  program  objectives  during  all  phases  [of 
the  System  Acquisition  Process]  requires  team  work  and 
support  from  all  available  sources.  Some  of  the  tasks 
...  are  performed  by  personnel  assigned  to 
organizations  other  than  the  program  office  and  some 
are  performed  before  a  program  office  is  organized. 
Other  tasks  involve  normal  staff  support  vital  to 
program  resources  such  as  financial,  personnel, 
procurement,  and  so  forth.  After  a  program  office  is 
formed,  the  program  manager  is  the  focal  point  for 
leadership  of  team  efforts  concerning  the  program.  No 
program  office  is  completely  self-suf  ficiei  .  Program 
managers  should  take  full  advantage  of  skills  available 
in  staff  offices  and  coordinate  efforts  of  related 
activities  [21:p.l-9], 

Technical  Order  Managers  (TOMAs)  are  not  the  only 
personnel  involved  in  the  technical  order  acquisition 
process.  Some  others  identified  by  regulation  are  the 
Program  Manager  (PM)  (14:p.2-l),  the  Deputy  Program  Manager 
for  Logistics  (DPML)  (22:3),  the  Integrated  Logistics  Sup- 
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port  Manager  (ILSM)  (23:2),  and  the  functional  divisions 
within  the  System  Program  Office  (SPO)  (21: fig. 10-2). 

Using  and  supporting  commands  are  also  involved  in  the 
TO  acquisition  process.  AFALDR  800-2,  DPML/ILSO  Responsibil¬ 
ities  and  Management  Indicators  (24:3),  states:  "It  is 
mandatory  that  the  data  requirements  [TOs]  be  coordinated 
with  the  using  and  supporting  commands  prior  to  being 
included  in  contract  requirements." 

Although  the  TO  manager  is  responsible  for  the 
acquisition  of  TOs,  he  or  she  should  use  the  using 
commands  to  plan  and  accomplish  the  acquisition 
program.  This  will  require  setting  up  a  point  of 
contact  at  the  using  command,  ATC,  and  the  ALC 
designated  as  the  system  manager  or  item  manager 
(SM/IM)  to  act  as  the  organizational  representative 
[ 14 :p. 29-1] . 

Air  Force  Logistics  Command  should  also  be  involved 
in  the  coordination  effort  within  the  TO  acquisition 
process: 

AFSC  is  responsible  for  technical  publications  during 
the  conceptual,  [demonstration/validation],  and 
acquisition  phases  of  the  system  life  cycle.  During  the 
operational  phase  (after  PMRT),  AFLC  assumes 
responsibility  for  these  publications  [25:5]. 

AF  Acquisition  Logistics  Center  (AFALC),  formerly 

AFALD,  is  also  involved  in  the  coordinating  process.  AFALD 

Regulation  800-2,  DPML/ILSO  Responsibilities  and  Management 

Indicators  (24:2),  directs:  "AFALD  staff  agencies  will 

support  the  DPML/ILSO  in  carrying  out  the  responsibilities 

identified  herein." 

The  role  of  the  Air  Force  Acquisition  Logistics  Center 
(formerly  AFALD)  in  the  technical  order  acquisition  process 


was  identified  by  the  USAF  Centralized  Technical  Order 

Management  Executive  Committee  (16:6) s 

AFALD/PTL  gives  help  and  guidance  to  the  DPML/ILSM  and 
TOMAs  as  required#  in  structuring  the  TO  program.  Help 
to  determine  the  type  and  format  of  TOs  is  included.  Mr 
Stiegmann  will  contact  AFALD  pertaining  to  their  role 
in  assisting  with  TO  formats. 

The  TO  acquisition  system  is  a  part  of  the  system 
acquisition  program.  A  high  degree  of  coordination  and 
communication  is  required  between  the  TO  acquisition  ele¬ 
ments  and  other  system  acquisition  elements. 

At  the  November  1983  meeting  of  tdie  Centralized  Tech¬ 
nical  Order  Management  Group  Executive  Committee  (17) ,  the 
representative  from  the  AF  Logistics  Management  Center 
presented  the  preliminary  results  of  a  survey  of  TO  Mana¬ 
gers.  The  survey  included  individual  interviews  with  17 
TOMAs#  6  supervisors,  and  a  group  interview  with  24  -TOMAs. 
The  preliminary  results  indicated  that  those  interviewed 
felt  a  major  problem  was  the  unresponsiveness  of  other 
acquisition  and  using  organizations  to  TO  acquisition  prob¬ 
lems.  Those  interviewed  also  felt  that  there  was  a  lack  of 
"clout"  within  the  TO  acquisition  process.  The  discussion  of 
these  problems  identified  a  lack  of  early  TO  acquisition 
planning,  and  coordination  between  the  various  system 
acquisition  agencies  as  the  primary  causes  of  these 
problems . 

The  CTOM  members  (17)  also  believed  that  the  coordi¬ 
nation  with  other  acquisition  agencies  is  inadequate  to 
properly  acquire  technical  orders. 
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Manning .  The  AF  Logistics  Management  Center 
interviews  with  technical  order  acquisition  personnel 
indicated  that  those  interviewed  considered  manning  to  be 
the  number  one  problem  facing  the  technical  order  acqui¬ 
sition  process  (17).  They  felt  that  there  is  no  standard  or 
criteria  for  assigning  technical  order  managers  to  a  pro¬ 
gram.  In  fact,  many  programs  do  not  have  full-time  technical 
order  managers . 

Research  by  Williams  and  Winn  (9)  supports  the  obser¬ 
vations  of  the  technical  order  managers  interviewed  by  the 
AF  Logistics  Management  Center: 

Overall  manning  of  the  ILS  [Integrated  Logistics 
Support]  effort,  including  the  Technical  Data  element, 
is  significantly  below  that  requested  [9:10]. 

Inadequate  .manning  is  in  itself  a  problem,  and  is  also 
a  major  contributor  to  the  other  technical  order  acquisition 
problems.  Earlier  planning  in  the  technical  order  acqui¬ 
sition  process  requires  adequate  manpower  accomplish  the 
job.  Better  coordination  and  communication  with  other 
agencies  also  requires  manpower. 

Training .  Mr.  Wilton  Stiegmann,  HQ  USAF/LEYE,  is 
the  chairperson  of  the  Centralized  Technical  Order  Manage¬ 
ment  (CTOM)  Executive  Committee.  He  stated: 

Corporate  memory  in  the  technical  order  acquisition 
process  is  very  low.  There  is  no  formal  career  field 
for  technical  order  acquisition  managers.  Personnel 
performing  duties  as  technical  order  acquisition 
managers  usually  have  little  or  no  experience  in  the 
field  prior  to  being  assigned  to  a  specific  project. 
They  learn  through  trial  and  error,  then  they  are 
transferred  and  their  knowledge  goes  with  them  [12]. 


A  problem  identified  in  the  AF  Logistics  Management 
Center  Survey  of  TOMAs  (November  1983  )  was  the  difficulty 
they  were  having  getting  TO  requirements  into  a  contract 
(17).  This  problem  was  identified  primarily  as  a  train¬ 
ing/education  problem.  There  are  170  reference  documents 
(17)  needed  to  compile  a  good  TO  acquisition  contract.  These 
reference  documents  include  TO  acquisition  regulations,  TO 
MIL  Specs,  and  many  others.  The  TOMAs  felt  that  many  refer¬ 
ences  were  outdated,  vague,  and  confusing.  One  of  the  tasks 
of  the  Centralized  Technical  Order  Management  Executive 
Committee  Meeting  (17)  was  to  get  AF  vide  coordination  on 
several  TO  acquisition  regulations.  The  problem  with  MIL 

i 

Standards  is  primarily  a  training  problem. 

Anr  interview  with  Mr.  John  Winters  from  the  Defense 
Material  Specifications  and  Standards  Office  (7)  added  one 

I 

I 

more  dimension  to  the  problem  with  contracts  and  MIL 
Standards.  MIL  Standards  are  dynamic  and  are  continually 

being  revised  and  changed.  Intensive  research  and  training 

j  ■ 

is  required  to  know  the  status  of  the  MIL  Standards  being 
used  to  acquire  TOs.  With  the  manpower  and  training  limita¬ 
tions  put  on  the  technical  order  manager  the  desired  level 
of  MIL  Standard  knowledge  is  not  always  achieved. 

The  Deputy  Secretary  of  Defense  Carlucci  Memorandum 
identified  the  need  for  increased  training  of  acquisition 
personnel : 

USDRE  should  consider  utilizing  a  number  of  creative 
techniques  to  translate  the  intent  of  these 
recommendations  to  all  levels.  This  could  include 
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formal  training  sessions/  conferences/  video  taped 
training  films/  articles/  and  policy  letters 
[ 5  tp. 9-172 ] . 

Formal  training  does  exist.  The  Air  Force  Institute  of 
Technology  has  a  two-week  training  course  for  technical 
order  acquisition  managers.  The  program  is  only  two  years 
old  and  as  more  graduates  reach  the  field/  there  should  be  a 
considerable  improvement  in  the  knowledge  level  of  technical 
order  acquisition  managers.  However/  one  two-week  course  is 
not  enough  (18).  The  CTOM  members  (17)  feel  that  "corporate 
memory"  is  a  vital  key  to  training  technical  order  acqui¬ 
sition  personnel.  Many  technical  order  acquisition  personnel 
are  not  able  to  attend  the  formal  training/  and  those  that 
do  often  work  in  technical  order  acquisition  long  before 
they  attend  the  training  (18).  The  CTOM  members  and  the 
technical  order  managers  interviewed  (17)  stressed  the  need 
for  individual  assistance  from  some  form  of  "assistance 
agency."  Mr  Stiegmann  (12)  added  that  the  Navy  does  assist 
their  technical  order  managers  with  just  such  an  "assistance 
agency"  (to  be  discussed  later).  He  further  commented  that 
such  specialized  assistance  is  needed  in  addition  to  in¬ 
creasing  formal  training. 

Mr.  Munguia  (18)/  the  course  director  for  the  Air  Force 
Institute  of  Technology  Technical  Order  Acquisition  Manage¬ 
ment  Course/  summarized  the  general  training  problem: 

The  knowledge  and  expertise  required  to  effectively 
manage  a  technical  order  acquisition  program  is 
immense.  In  a  two  week  course,  I  can  not  do  much  more 
than  give  an  overview  of  the  major  topics.  The  best  the 
students  can  do  is  learn  the  scope  of  their 
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responsibilities  and  hopefully  where  to  go  to  when  they 
need  assistance  with  the  details.  Unfortunately,  right 
now  there  is  really  no  place  they  can  go  to  get  the 
assistance  they  need.  They  generally  turn  to  other 
technical  order  acquisition  managers  who  may  or  may  not 
have  the  correct  information  for  them.  Each  program  has 
unique  needs  and  what  was  good  for  one  may  not  be  good 
for  another.  They  need  experts  and  specialists  to  turn 
to  for  assistance  [18]. 

Remarks .  There  do  appear  to  be  problems  with  the 
technical  order  acquisition  process.  Technical  order  plan¬ 
ning  needs  to  begin  earlier  in  the  acquisition  cycle.  Better 
coordination  and  communication  between  all  agencies  involved 
in  the  weapon  system  acquisition  process  is  needed.  The 
problems  will  require  more  and  better  trained  manpower  be- 
•  fore  they  can  be  solved.  The  next  section  will  discuss 
changes  to  the  technical  order  acquisition  process  that  have 
been  recommended  to  solve  those  problems. 

Recommended  Changes 

Recommendations  for  improving  the  technical  order 
acquisition  system  have  come  from  a  variety  of  sources.  The 
recent  concern  for  life  cycle  cost  considerations  and  an 
increased  interest  in  maintainability  and  reliability  in  the 
system  acquisition  program  have  resulted  in  new  system 
objective  priorities  which  require  changes  to  the  technical 
order  acquisition  process. 

Recommended  changes  to  the  technical  order  acquisition 
process  will  be  discussed  as  they  relate  to  the  four  problem 
areas  identified  in  the  preceding  section  of  this  literature 
review.  Those  four  problem  areas  ares 
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1.  Need  for  earlier  planning. 

2.  Better  coordination  and  communication. 

3.  Manning. 

4.  Training. 

The  recommended  changes  are  not  mutually  exclusive.  The 
technical  order  acquisition  process  is  an  embedded  system 
within  the  system  acquisition  process.  As  a  result/  there 
are  no  cut  and  dry  dividing  lines  between  activities  within 
the  system.  In  fact,  there  are  no  cut  and  dry  dividing  lines 
between  the  four  problems  identified  above.  For  example,  the 
need  for  earlier  planning  is  partially  a  result  of  a  lack  of 
coordination  between  acquisition  agencies.  The  lack  of 
coordination  is  partially  a  result  of  the  lack  of  training 
of  the  participants  and  partially  the  result  of  not  having 
enough  manpower  to  affect  the  required  coordination.  Some 
recommended  changes  will  affect  more  than  one  problem. 

Need  for  Earlier  Planning .  Technical  order  planning 
does  not  begin  early  enough  in  the  acquisition  cycle. 
Recommended  changes  to  solve  this  problem  include: 

1 .  Establishment  of  a  technical  order  management 
center . 

2.  Establishment  of  "skeleton"  Statement  Of  Needs 
(SONs),  Requests  For  Proposals  (RFPs),  and 
contracts . 

3.  Establishment  of  technical  orders  (operations  and 
maintenance  manuals)  as  a  "product"  to  be  included 
as  a  separate  line  item  in  the  SON,RFP,  and 
contract. 

Technical  Order  Management  Center.  Interviews 
with  technical  order  acquisition  personnel  (17)  indicate  a 
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need  for  a  central  Air  Force  agency  (tentatively  called  Air 
Force  technical  Order  Center)  which  could  assist  technical 
order  acquisition  personnel  with  problems  confronting  them 
during  technical  order  acquisition. 

The  members  of  the  CTOM  committee  also  recommended  the 
establishment  of  an  Air  Force  Technical  Order  Center 
(AFTOC).  They  have  established  a  work  group  to  recommend 
several  alternative  centralized  technical  order  management 
organizations  to  the  Air  Staff  (6:5). 

The  Navy  uses  a  centralized  management  agency  system. 
For  aircraft  acquisitions,  the  Naval  Air  Technical  Service 
Facility  (NAVAIRTECHSERVFAC)  performs  the  centralized  tech¬ 
nical  order  management  function.  Mr.  Tom  Martin,  from 
NAVAIRTECHSERVFAC,  stated  that  his  organization  is  heavily 
involved  in  the  early  conceptual  phases  of  the  acquisition 
process.  Their  specialists  assist  in  early  planning  and 
procurement  actions  affecting  technical  orders.  Some  early 
planning  responsibilities  of  the  NAVAIRTECHSERVFAC  are 
identified  in  NAVAIRINST  5600. 20B  (26:11): 

1.  Coordinate  and  maintain  TM  plans  that  shall  provide 
for  coverage  of  each  complete  system  and  its  related 
equipment  and  components. 

2.  Prepare  TM  requirements  for  inclusion  in 
contracts,  procurement  requests,  AIRTASKS, 
project  orders  or  work  requests,  as  required,  and 
place  orders  through  proper  contracting  channels. 

3.  In  consonance  with  policy  direction  from  the  TM 
program  coordinator,  develop  procedures  for 
effective  TM  project  management. 
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The  Army  also  uses  a  centralized  technical  order  man¬ 
agement  agency.  The  Army  system  is  being  used  as  a  model  for 
the  CTOM  work  group  recommendations. 

A  centralized  technical  order  management  center  has 
been  recommended  to  provide  earlier  technical  order  planning 
assistance  to  acquisition  programs. 

Skeleton  Documents.  The  recommendation  for  skele¬ 
ton  documents  to  assist  technical  order  acquisition  person¬ 
nel  was  made  by  the  Air  Force  Inspector  General  team  in  a 
briefing  to  the  CTOM  committee  (17).  Statement  of  Needs 
(SONs)  identify  operational  needs  to  meet  a  threat.  The  SON 
for  a  project  is  the  first  step  in  the  acquisition  of  a  new 
system.  The  inclusion  of  technical  order  needs  in  a  SON  will 
facilitate  earlier  technical  order  planning.  The  SONs  are 

a 

developed  before  an  official  Systems  Program  Office  is 
formed »  therefore,  a  "skeleton"  of  the  technical  order  needs 
that  could  be  included  in  a  SON  could  be  a  help  to  the 
personnel  writing  the  SON.  The  Request  for  Proposal  (RFP)  is 
the  document  that  gets  the  contractor  involved  in  the 
process.  A  "skeleton"  of  the  possible  technical  order  needs 
could  help  the  personnel  writing  the  RFP  make  the  contractor 
aware  of  those  needs  from  the  beginning.  A  "skeleton"  of 
items  that  should  be  considered  when  writing  a  contract 
could  also  be  useful. 

"Skeleton"  technical  order  requirements  for  these 
acquisition  documents  can  help  early  planners  identify  tech- 
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nical  order  needs  in  early  phases  of  the  acquisition 
process. 

Technical  Orders  as  a  Product .  The  Air  Force 
purchases  "data"  which  includes  all  the  engineering  drawings 
and  blueprints  used  to  develop  the  system  as  well  as  the 
operations  and  maintenance  manuals. 

Mr  John  Winters  (7) ,  from  the  Defense  Material  Specifi¬ 
cations  and  Standards  Office  recommended  that  the  Air  Force 
separate  manuals  (TOs)  from  data  in  the  acquisition  process. 
He  said  that  the  Army  and  Navy  purchase  technical  manuals  as 
a  separate  line  item  product.  He  believed  that  as  a  sepa¬ 
rate  line  item,  technical  order  needs  would  be  more  visible 
to  early  planners  and  the  contractor.  Including  technical 
orders  as  a  separate  line  item  would  force  planners  to 
focus  attention  on  technical  orders  much  earlier  than  they 
do  now. 

Mr  Stiegmann  (12),  HQ  USAF/LEYE,  expressed  similar 
thoughts  on  the  subject. 

Remarks.  The  recommendations  for  getting  earlier 
planning  for  technical  order  acquisition  includes  the 
establishment  of  a  centralized  technical  order  management 
agency  and  the  development  of  "skeleton"  documents  to  assist 
early  planners.  The  inclusion  of  technical  orders  as  a  sepa¬ 
rate  line  item  product  was  also  suggested  to  force  earlier 
attention  to  technical  orders.  The  three  recommendations  are 
complementary  to  each  other  and  combined  with  better  coordi¬ 
nation  and  communication  between  acquisition  agencies,  as 
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veil  as  improved  manning  and  training  of  technical  order 
acquisition  personnel/  should  result  in  earlier  planning  of 
technical  order  acquisitions  (12). 

Better  Coordination  and  Communicat ion .  Better  coordi¬ 

nation  and  communication  between  technical  order  acquisition 
agencies  and  other  acquisition  agencies  is  needed.  Recom¬ 
mendations  to  solve  this  problem  are: 

1.  Establishment  of  a  technical  order  management 
center . 

2.  Establishment  of  " skeleton"  Statement  Of  Needs 
(SONs)/  Requests  For  Proposals  (RFPs)/  and 
contracts . 

j 

3.  Establishment  of  a  handbook  identifying 
coordination/cominunication  responsibilities  • 

Technical  Order  Management  Center.  The  technical 

| 

order  acquisition  management  center  recommended  in  the 

preceding  section  of  this  thesis  can  also  help  improve  coor- 

i 

dination  and  communication  between  technical  order  acqui- 

| 

sition  personnel  and  other  acquisition  agencies (6).  As  a 

j 

center  of  specialized  technical  order  acquisition  knowledge/ 
the  technical  order  management  center  can  assist  acquisition 
personnel  with  technical  order  information.  The  center  will 
be  especially  important  to  assist  in  the  early  phases  of  the 
acquisition  process  prior  to  the  assignment  of  technical 
order  management  personnel  to  a  program.  An  important  re¬ 
sponsibility  assigned  to  the  NAVAIRTECSERVFAC  is  to  insure 
that  proper  coordination  of  technical  order  requirements  is 
accomplished  (26).  A  technical  order  management  center  could 


perform  a  similar  function  for  the  Air  Force  to  help  improve 
coordination  and  communication. 

Skeleton  Documents .  The  recommendation  for 
"skeleton"  documents  has  been  discussed  in  detail  in  a 
preceding  section.  The  Statement  Of  Need  (SON)  and  Request 
For  Proposal  (RFP)  are  generally  accomplished  prior  to  the 
assignment  of  technical  order  managers  to  the  acquisition 
program.  "Skeleton"  documents  can  help  improve  coordination 
and  communication  with  other  acquisition  agencies  prior  to 
the  assignment  of  a  technical  order  manager  to  a  project. 

The  skeleton  document  vould  be  a  starting  point  for  plan¬ 
ning,  but  a  point  of  contact  would  also  have  to  be  estab¬ 
lished  to  assist  individual  programs  with  the  use  of  the 
documents  (18). 

Handbook .  The  recommendation  for  a  handbook  to 
improve  the  coordination  and  communication  between  the 
acquisition  personnel  and  other  agencies  came  from  Mr. 
Munguia  (18),  the  course  director  for  the  technical  order 
acquisition  management  course  at  AFIT.  Feedback  from  his 
students  indicates  that  the  amount  of  coordination  required 
for  technical  order  acquisition  is  overwhelming.  He  felt 
they  would  benefit  from  a  handbook  dedicated  specifically  to 
identifying  the  required  coordination  and  communication 
channels.  Mr  Stiegmann  (12),  HQ  USAF/LEYE,  added  that  such  a 
handbook  should  be  of  value  to  the  other  acquisition  agen¬ 
cies  involved  with  technical  order  acquisition  also.  They 


could  see  what  their  coordination  and  communication  respon¬ 
sibilities  are. 

Remarks.  Recommendations  to  improve  coordination 
and  communication  include  a  technical  order  management 
center ,  "skeleton"  documents/  and  a  handbook/  as  well  as 
improving  manning  and  training  of  technical  order  personnel. 

Manning .  Recommendations  to  improve  manning  problems 
in  the  technical  order  acquisition  field  include: 

1.  Establishment  of  a  separate  AFSC  for  technical  order 
managers. 

2.  Establishment  of  a  technical  order  management 
center* 

Separate  AFSC.  The  CTOM  committee  (27:6)  recom¬ 
mended  the  establishment  of  a  separate  AF  Speciality  Code 
(AFSC)  for  technical  order  managers.  Without  a  separate 
AFSC,  there  is  no  continuity  "pipeline"  for  technical  order 
management  personnel.  Technical  order  managers  are  generally 
on  the  job  just  long  enough  to  become  proficient;  then  they 
return  to  their  previous  AFSC  (12). 

Technical  Order  Management  Center.  The  recom¬ 
mendation  of  the  technical  order  management  center  has  been 
discussed  in  a  previous  section  of  this  thesis.  The  estab¬ 
lishment  of  a  technical  order  management  center  will  allow 
smaller  programs  to  effectively  purchase  technical  orders 
without  the  assignment  of  a  technical  order  manager.  The 
technical  order  management  center  can  assist  in  technical 
order  acquisition  needs  of  those  smaller  programs/  thus 
reducing  the  need  for  technical  order  management  manpower  in 
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those  programs.  The  end  result  will  be  a  reduction  of  man¬ 
power  requirements  over-all  (18). 

Remarks .  The  recommendations  aimed  at  solving 

manning  problems  are  complementary.  The  establishment  of  a 
separate  AFSC  for  technical  order  managers  is  aimed  at 
retention#  and  the  technical  order  management  center  is 
aimed  at  reducing  the  over-all  manpower  requirement. 

Training.  Recommendations  aimed  at  solving  the  train¬ 
ing  problem  are: 

1.  Separate  AFSC  for  technical  order  managers. 

2.  Technical  order  management  center. 

3.  Handbook. 

Separate  AFSC.  The  recommendation  for  a  separate 
AFSC  for  technical  order  managers  has  already  been  dis¬ 
cussed.  A  separate  AFSC  will  help  retain  trained  technical 
order  managers  which  will  result  in  a  higher  average  train¬ 
ing  level. 

Technical  Order  Management  Center.  The  recommen¬ 
dation  for  the  technical  -  order  management  center  has  already 
been  discussed.  The  management  center  can  assist  technical 
order  managers  with  specific  problems.  That  assistance  will 
increase  the  knowledge  of  the  technical  order  managers. 

Handbook.  The  recommendation  for  a  handbook  on 
coordination  and  communication  channels  between  technical 
order  acquisition  personnel  and  other  acquisition  agencies 
has  already  been  discussed.  Such  a  handbook  can  help  in¬ 
crease  the  knowledge  of  technical  order  managers. 
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Remarks .  The  recommendations  to  improve  the 
training  level  of  technical  order  managers  are  comple¬ 
mentary.  The  establishment  of  a  separate  AFSC  will  help  keep 
trained  technical  order  managers.  The  technical  order 
management  center  and  the  handbook  will  help  increase  the 
knowledge  of  technical  order  managers. 


Chapter  Summa] 


User-oriented  problems  of  inadequate  technical  orders 


have  been  identified.  Problems  with  the  technical  order 


acquisition  process  that  could  result  in  the  acquisition  of 
inadequate  technical  orders  have  also  been  identified,  as 
well  as  recommended  changes  to  the  technical  order  acqui¬ 
sition  process  aimed  at  solving  those  problems. 

The  technical  order  acquisition  process  problems  are: 

1.  Technical  order  planning  does  not  begin  early  enough 
in  the  acquisition  cycle. 

2.  Coordination  and  communication  between  all  agencies 
involved  in  the  weapon  system  acquisition  process  is 
not  adequate. 

3.  Manpower  dedicated  to  technical  order  acquisition  is 
not  adequate. 

4.  Training  and  assistance  for  technical  order 
acquisition  personnel  is  not  adequate. 

The  recommended  changes  to  the  technical  order  acqui¬ 
sition  process  are:  \ 

1.  The  establishment  of  a  technical  order  management 
center . 

2.  "Skeleton"  Statement  Of  Needs  (sONs),  Requests  For 
Proposals  (RFPs),  and  contracts. 
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3.  Treat  technical  orders  as  a  "product"  to  be  included 
as  a  separate  line  item  in  the  SON,  RFP,  and 
contract. 

4.  Handbook  identifying  coordination/conmunication 
responsibilities. 

5.  Separate  AFSC  for  technical  order  managers. 

The  objective  of  this  research  project  is  to  answer 
four  research  questions  that  relate  to  the  four  problems 
identified  with  the  technical  order  acquisition  process. 
Those  four  research  questions  are: 

1.  How  can  technical  order  acquisitions  be  integrated 
into  earlier  phases  of  the  weapon  system  acquisition 
cycle? 

2.  How  can  communication  and  coordination  between 
technical  order  acquisition  agencies  and  other 
acquisition  agencies  be  improved? 

3.  How  can  technical  order  acquisition  personnel 
manning  and  retention  be  improved? 

ft 

4.  How  can  training  and’ knowledge  of  technical  order 
acquisition  personnel  be  improved? 

These  four  questions  will  be  answered  by  testing  the 
validity  and  completeness  of  the  recommended  changes  to  the 
technical  order  acquisition  process  that  have  been  identi¬ 
fied.  Those  recommendations  that  are  found  to  be  valid  will 
in  effect  be  hypotheses  that,  once  implemented  and  tested, 
will  either  solve  the  problem  or  lead  to  further  hypotheses 
to  solve  the  problems. 
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III.  Methodology 


Justification  of  Research  Method 

It  was  noted  in  the  introduction  that  because  this 
research  was  exploratory  in  nature ,  no  hypotheses  were  to  be 
tested.  Rather ,  problems  with  the  technical  order  acqui- 
sition  process  were  identified  in  the  literature  review/ 
along  with  recommended  solutions  to  those  problems.  The 
purpose  of  the  research  phase  was  to  assess  the  validity  and 
completeness  of  the  problems  and  recommended  solutions. 

This  assessment  was  accomplished  through  telephone 
interviews  with  personnel  active  in  the  TO  acquisition  pro¬ 
cess.  The  respondents  were  asked  to  express  their  agreement 

% 

or  disagreement  with  the  problems  and  recommended  solutions 
presented  to  them  during  the  interview.  The  interview  sched¬ 
ule  is  presented  as  Appendix  A.  The  magnitudes  of  the  re¬ 
spondents'  agreement  or  disagreement  with  the  identified 
problems  and  recommended  solutions  were  used  to  determine 
the  validity  of  those  problems  and  solutions  to  accurately 
reflect  the  actual  conditions  of  the  TO  acquisition  process. 
In  addition,  open-ended  questions  were  asked  to  determine  if 
the  identified  problems  and  recommended  solutions  repre¬ 
sented  a  complete  list. 

Structured  questions  were  used  for  the  interviews;  how¬ 
ever,  the  inherent  flexibility  of  personal  interrogation 
allowed  for  amplification  of  responses  as  necessary. 
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The  versatility  of  this  method  is  its  greatest 
strength.  It  is  the  only  practical  vay  to  learn  many 
types  of  information  and  the  most  economical  vay  in 
many  other  situations  [28:213]. 

Alt no ugh  several  of  the  personnel  interviewed  were 
located  on  the  same  installation  as  the  researchers,  tele¬ 
phone  interviews  were  used  in  all  cases.  This  was  done  to 
eliminate  possible  response  errors  due  to  one  group  re¬ 
ceiving  personal  interviews  and  the  other  telephone 
interviews . 

Population  Description 

The  subject  of  this  research  was  the  technical  order 
acquisition  process.  Four  product  divisions  of  the  Air  Force 
Systems  Command  (AFSC)  dealt  with  the  acquisition  of  major 
AF  weapon  systems  and  related  equipment,  and  thus  were 
within  the  scope  of  this  research.  They  were  the  Aero¬ 
nautical  Systems  Division  (ASD),  the  Electronics  Systems 
Division  (ESD),  the  Armament  Division  (AD),  and  the  Bal¬ 
listic  Missile  Office  (BMO).  The  fifth  product  division  in 
AFSC,  Space  Division,  was  not  included  in  the  research  be¬ 
cause  of  the  unique,  non- weapon  system  nature  of  the  equip¬ 
ment  acquired  by  that  division. 

A  description  of  the  organizational  structures  of  the 
four  divisions  is  necessary  to  understand  the  nature  of  the 
population.  These  structures  differ  because  of  the  varying 
scope  of  the  programs  for  which  the  divisions  are  respon¬ 
sible.  For  example,  ASD  deals  with  major  systems  acqui¬ 
sitions  such  as  aircraft,  simulators,  engines,  and  related 
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aeronautical  equipment/  vhile  ESD  and  AO  deal  vith  smaller 
veapon  systems  or  subsystems  of  larger  systems.  What  follows 
is  a  general  description  of  the  different  divisions'  System 
Program  Office  (SPO)  organizations/  recognizing  that  each 
SPO  will  differ  in  detail. 

In  ASD/  individual  SPOs  are  often  large  organizations/ 
with  separate  offices  dedicated  to  specific  acquisition 
responsibilities/  and  these  separate  offices  contain  spe¬ 
cialists  in  the  corresponding  areas.  In  general/  each  ASD 
SPO  has  three  levels  of  responsibility  regarding  TO  acqui¬ 
sition.  At  the  upper  level  are  Program  Managers  (PMs)/  who 
are  responsible  for  the  acquisition  of  the  entire  system, 
including  TOs.  Reporting  to  the  PM  is  a  Deputy  Program 
Manager  for  Logistics  ( DPML ) ,  who  is  responsible  for  the 
portion  of  system  acquisition  that  relates  to  logistics, 
again  including  TOs.  Finally,  there  are  Technical  Order 
Managers  (TOMAs),  reporting  to  the  DPML,  that  coordinate  and 
manage  the  day-to-day  activities  of  TO  acquisition. 

In  contrast  to  the  large-scale  SPOs  typical  of  ASD  are 
the  SPOs  of  ESD  and  AD.  The  systems  these  divisions  are 
responsible  for  acquiring  are  typically  smaller  in  scope 
than  those  of  ASD,  with  a  corresponding  reduction  of  ded¬ 
icated  functions  within  those  SPOs.  Neither  division  has 
TOMAs  per  se>  in  ESD  the  DPMLs  perform  TO  acquisition  as 
part  of  their  normal  duties,  while  in  AD  TO  acquisition 
specialists  are  matrixed  from  a  central  office  into  SPOs  as 
required.  The  organization  of  BMO  is  a  mix  of  the  other 
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three;  the  SPOs  are  large  (as  in  ASD)  but  few  in  number, 
while  TO  acquisition  personnel  are  matrixed  from  a  central 
office  similar  to  the  ESD  and  AD  structure. 

The  personnel  interviewed  for  this  research  were  those 
within  the  four  product  divisions  who  are  involved  in  TO 
acquisition,  and  members  of  the  USAF  Centralized  TO 
Management  (CTOM)  Group.  The  CTOM,  comprised  of  TO  special¬ 
ists  from  the  major  commands,  was  described  in  the  intro¬ 
duction  to  this  report. 

For  each  product  division,  respondents  were  grouped 
according  to  their  duty  category,  rather  than  strictly  by 
their  title.  A  list  of  all  the  current  systems  active  in  the 
four  product  divisions  was  provided  by  the  Basis  Production 
System,  a  data  base  maintained  by  the  Air  Force  Acquisition 
Logistics  Center  at  Wright-Patterson  AFB.  This  list. included 
the  names  and  telephone  numbers  of  the  PPML  for  each  system, 
from  which  duplicates  were  eliminated,  and  the  list  was  then 
used  as  a  starting  point  for  interviews. 

When  conducting  the  survey  with  each  DPML,  the  inter¬ 
viewer  requested  the  name  and  telephone  number  of  their 
respective  PM  and  TO  specialist.  By  this  technique  the 
respondents  in  1ESD,  AD  and  BMO  were  functionally  grouped 
into  the  three  ^asic  levels  of  responsibility  seen  in  the  TO 
acquisition  process:  overall  managers,  logistics  managers, 
and  TO  acquisition  specialists.  In  this  manner  the  list  of 
potential  respondents  grew  from  only  DPMLs  to  include  the 
duty  categories  of  PM  and  TOMA. 
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The  intent  of  the  research  was  to  conduct  a  census 
rather  than  a  sample  of  personnel  involved  in  TO  acqui¬ 
sition.  This  was  possible  because  of  the  relatively  small 
size  of  the  population.  Of  the  original  54  DPMLs  identified , 
49  DPML  interviews  resulted,  43  of  which  were  with  the 
original  DPMLs  (or  their  replacements).  The  variation  in  the 
responses  from  the  original  DPML  list  came  as  a  result  of 
personnel  transfers  without  replacements,  and  SPO  reorgan¬ 
izations.  These  statistics  are  discussed  here  to  point  out 
the  difficulties  encountered  in  conducting  a  census. 

The  identification  of  the  PM  and  TOMA  population  pre¬ 
sented  unique  problems.  The  total  number  of  PM  and  TOMA 
personnel  is  unknown,  and  efforts  to  determine  those  totals 
were  complicated  by  the  method  of  grouping  by  duty  category 
rather  than  by  title.  Thus,  the  fact  that  seventeen  PMs  and 
fifty -three  TOMAs  were  interviewed  cannot  be  related  to  a 


possible  total.  The 

authors 

assumed 

that  the 

PMs  and  TOMAs 

identified  by  the  DPMLs  were 

in  those  positions. 

al 1  the 

personnel 

who  function 

The  following  is 

category  and  division 

a  list 

• 

• 

of  the 

respondents  in  each 

Division 

PM 

DPML 

TOMA 

Total 

ASD 

13 

21 

27 

61 

ESD 

2 

18 

17 

37 

AD 

1 

7 

7 

15 

BMO 

1 

3 

2 

6 

Total 

17 

49 

53 

119 
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In  addition/  II  CTOM  members  were  interviewed.  This 
elevated  the  total  number  if  telephone  interviews  to  130. 

The  Telephone  Interview 

The  method  of  interrogation  was  a  telephone  interview 
in  all  cases/  with  the  same  interview  schedule  used  for  each 
interview.  The  instrument  (see  Appendix  A)  began  with  an 
introduction  by  the  interviewer  to  the  respondent,  explain¬ 
ing  the  purpose  of  the  interview  and  giving  a  brief  overview 
of  the  research. 

The  first  questions  were  open-ended,  or  overhead, 
designed  to  learn  of  any  problems  or  solutions  to  problems 
that  the  respondent  believed  were  important.  The  remainder 
of  +*he  questions  dealt  with  the  specific  problems  and  recom¬ 
mended  solutions  that  this  research  was  attempting  to  as¬ 
sess.  The  interviewer  requested  that  responses  to  the  later 
questions  be  either  "yes"  or  "no">  however,  the  interviewer 
noted  any  additional  comments. 

The  respondents  were  considered  to  be  independent  of 
each  other.  Thus  no  specific  sequence  of  interviews  was 
followed.  This  allowed  other  interviews  to  be  conducted  if  a 
particular  respondent  could  not  be  contacted  for  some 
period.  No  information  will  be  presented  that  links  an 
individual  to  a  specific  response. 

Research  Methodology 

An  explanation  of  how  the  results  of  the  telephone 
interviews  were  used  for  thi3  research  follows.  The  actual 
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results  and  findings  will  be  discussed  in  later  chapters  of 
this  report . 

The  purpose  of  the  telephone  survey  was  to  assess  how 
accurately  the  previously  identified  TO  acquisition  process 
problems/  and  the  recommended  solutions  to  those  problems/ 
represented  the  views  of  those  people  involved  in  the  tech¬ 
nical  order  acquisition  process.  In  order  to  perform  this 
assessment  the  responses  of  the  four  product  divisions  were 
compared  with  each  other/  as  were  the  responses  of  the 
individual  duty  categories  (PM,  DPML,  TOMA  and  CTOM) .  Sig¬ 
nificant  disagreement  between  the  divisions  or  duty  cate¬ 
gories  on  individual  questions  was  evaluated  to  determine  if 
the  cause  of  the  discrepancy  was  related  to  the  inherent 
differences  in  the  groups,  or  caused  by  a  previously  undis- 
covered  problem  or  recommended  solution.  The  basis  for  this 
evaluation  was  the  additional  comments  made  by  the  respon¬ 
dents,  and  the  authors'  knowledge  of  the  differences  in  the 
groups . 

Validation  of  the  identified  problems  and  solutions  was 
determined  by  noting  whether  or  not  a  majority  of  the  re¬ 
spondents  agreed  with  the  stated  problems  and  solutions. 
Survey  questions  dealing  with  validation  were  constructed  to 
result  in  either  a  "yes"  or  "no"  response. 

Completeness  was  assessed  by  the  use  of  open-ended 
questions,  to  determine  if  any  areas  were  overlooked  in  the 
identification  of  either  TO  acquisition  problems  or  solut¬ 
ions  to  those  problems.  These  open-ended  questions  preceded 
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the  validation  questions  so  that  respondents  would  not  be 
led  to  answer  with  specific  problems  and  solutions  already 
in  mind.  Thus#  the  respondents  provided  their,  own  views  of 
problems ,  and  recommended  solutions  to  those  problems# 
without  prior  knowledge  of  the  research  questions. 

Responses  to  the  open-ended  questions  were  evaluated  to 
determine  if  they  fit  into  the  category  of  one  of  the  re¬ 
search  questions.  The  number  of  responses  that  supported 
this  research  was  then  compared  to  the  number  that  did  not. 
Those  that  did  not  were  further  evaluated  to  determine  if  a 
pattern  existed  that  justified  comment.  The  responses  to  the 
open-ended  questions  were  not  evaluated  separately  by  divis¬ 
ion  or  duty  category.  It  was  expected  with  these  questions 
that  individual  differences  were  more  likely  to  occur  than 
were  group  differences. 

These  are  the  guidelines  that  were  used  to  evaluate  the 
validity  and  completeness  of  the  problems  and  solutions 
being  assessed.  Where  differences  between  groups  were  found, 
they  were  used  to  help  to  explain  the  problems  or  to  justify 
the  solutions  being  tested.  Hie  differences  were  also  of 
some  value  in  determining  the  final  recommendations  of  this 
exploratory  study. 

The  simplicity  of  the  evaluation  guidelines  used  re¬ 
flects  the  exploratory  nature  of  this  research.  The  data 
gathered  represents  the  opinions  of  personnel  actively 
involved  in  the  TO  acquisition  process.  The  determination  of 
agreement  (or  disagreement)  with  the  identified  problems  and 
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solutions  was  intended  to  be  an  aid  to  those  charged  with 
the  improvement  of  the  TO  acquisition  process.  This  infor¬ 
mation  should  provide  confidence  that  the  implementation  of 
the  recommended  solutions  will  correct,  or  at  least  ease, 
the  current  problems. 

When  the  telephone  interviews  were  completed,  the  data 
were  compiled  by  division  and  duty  category.  Additional 
comments  to  the  "yes"  and  "no"  questions  were  grouped  in  a 
similar  fashion.  The  data  was  then  evaluated  in  the  method 
described  previously.  The  results  of  those  evaluations  are 
contained  in  the  following  chapter. 
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IV.  Results 


Chapter  Organization 

The  data  gathered  during  the  telephone  interview  was 
examined  according  to  the  purpose  of  the  questions.  The 
questions  which  tested  for  completeness  of  the  identified 
problems  and  recommended  solutions  ( 1  and  2 )  were  examined 
first.  Next,  the  questions  which  tested  for  the  validity  of 
the  research  questions  (3  through  6)  were  examined.  The  data 
was  viewed  in  two  perspectives:  grouped  by  product  division, 
and  grouped  by  duty  category.  Using  the  same  perspectives, 
the  questions  which  tested  for  the  validity  of  the  suggested 
solutions  (7  through  11)  was  then  examined.  The  results  will 
be  presented  in  either  a  table  format  with  the  narrative,  or 
in  an  appendix. 

The  data  from  interviews  with  members  of  the  CTOM  were 
not  grouped  with  either  a  particular  product  division  or 
duty  category.  Because  the  duties  of  the  CTOM  members  cut 
across  the  spectrum  of  the  TO  acquisition  process,  their 
responses  were  included  with  each  perspective,  for  each 
group  being  examined. 

Conclusions  regarding  the  responses  will  not  be  ad¬ 
dressed  in  this  chapter.  Chapter  V  will  discuss  the  results 
of  the  interview,  and  Chapter  VI  will  make  recommendations 
based  upon  the  gathered  data. 
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Completeness  of  the  Questions 

The  responses  to  question  1  and  2  were  qualitative 
rather  than  quantitative.  Therefore#  the  responses  are  pre¬ 
sented  rather  than  a  numerical  chart.  Appendix  B  is  a  com¬ 
plete  list  of  responses  to  Question  1#  and  Appendix  C  lists 
the  same  information  for  Question  2.  Each  appendix  displays 
the  responses  by  duty  category  within  each  product  division. 
The  responses  are  not  direct  quotes;  the  authors  synthesized 
the  responses  into  main  thoughts  in  order  to  improve  the 
readability  of  the  responses. 

Validity  of  the  Research  Questions 

The  responses#  by  product  division#  to  Question  3 
through  6  are  shown  in  Table  4.1#  while ‘Table  4.2  contains 
the  same  information  by  duty  category. 

Note  that  for  each  category  and  question  a  "%  Yes" 
accompanies  the  "Yes"  and  "No"  responses.  The  percentage  of 
positive  responses  for  a  particular  entry  was  calculated 
using  the  total  number  of  respondents  in  the  category.  This 
explains  cases  where  the  percentages  are  identical#  while 
the  indicated  number  of  responses  may  differ.  An  example  of 
this  is  in  Table  4.1#  the  AD  category#  Questions  4  and  6. 

The  data  given  by  division  and  by  duty  category  in  this 
chapter  are  totals  for  their  respective  groups  and  vere  used 
in  the  evaluations  discussed  in  the  following  chapter. 


47 


TABLE  4.1 


TABLE  4.2 


Questions  3  Through  6  -  By  Duty  Category 


|  Question 

Duty 

Category 


CT0M 

Yes 

No 

%  Yes 

PM 

Yes 

No 

%  Yes 

DPML 

Yes 

No 

%  Yes 

TOMA 

Yes 

No 

%  Yes 

12 


5 


70.6 


31 


18 


63.3 


30 


23 


56.6 


10 


41.2 


17 


29 


34.7 


11 


42 


20.8 


10 


9. 


9 


5 


52.9 


13 


36  * 


26.5 


10 


43 


18.9 


8 


23.5 


8 


40 


16.3 


11 


42 


20.8 


A  more  detailed  breakdown  of  the  responses  to  the  tele 
phone  interview  is  given  in  Appendixes  D,  E,  and  F,  where 
the  responses  for  questions  3  through  11  are  given  by  duty 
category  for  each  division. 


TABLE  4.3 


Questions  7  Through  11  -  By  Product  Division 
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V.  Conclusions 


The  objective  of  this  research  vas  to  identify  problems 
with  the  acquisition  of  U.  S.  Air  Force  technical  orders, 
and  to  identify  possible  changes  to  the  technical  order 
acquisition  process  that  will  solve  those  problems.  That 
objective  was  accomplished  by  researching  four  questions. 

The  results  of  that  research  will  be  used  to  answer  the 
research  questions  and  to  formulate  other  conclusions  about 
the  problems  of  the  technical  order  acquisition  process. 
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Answers  to  Research  Questions 


The  telephone  survey  that  was  accomplished  as  part  of 
this  research  resulted  in  information  that  could  be  used  to 
answer  the  four  research  questions  presented  earlier.  Those 
research  questions  are: 

1.  How  can  technical  order  acquisitions  be  integrated 
into  earlier  phases  of  the  weapon  system  acquisition 
cycle? 

2.  How  can  communication  and  coordination  between 
technical  order  acquisition  agencies  and  other 
acquisition  agencies  be  improved? 

3.  How  can  manning  and  retention  of  technical  order 
acquisition  personnel  be  improved? 

4.  How  can  training  and  knowledge  of  technical  order 
acquisition  personnel  be  improved? 

Each  of  these  questions  was  investigated  from  three 
perspectives.  The  first  perspective  was  an  assessment  of  the 
validity  of  the  problem  associated  with  each  research  ques- 


.•* 


53 


tion.  This  assessment  was  accomplished  by  analyzing  the  re¬ 
sponses  to  questions  3  through  6  of  the  telephone  survey. 
The  second  perspective  was  an  assessment  of  the  validity  of 
the  changes  to  the  technical  order  acquisition  process  that 
have  been  recommended  as  solutions  to  those  problems.  This 
assessment  was  accomplished  by  analyzing  the  responses  to 
questions  7  through  11  of  the  telephone  survey.  The  third 
perspective  was  an  assessment  of  a  relative  priority  for 
each  problem  and  recommended  solution.  This  assessment  was 
accomplished  by  analyzing  the  responses  to  questions  1  and  2 
of  the  telephone  survey. 

Each  research  question  will  be  answered  from  all  three 
perspectives . 

Research  Question  Number  1,.  The  first  research  ques¬ 
tion  vast 

1.  How  can  technical  order  acquisitions  be  integrated 
into  earlier  phases  of  the  weapon  system  acquisition 
cycle? 

Validity  of  the  Problem.  The  validity  of  the 
problem  associated  with  this  research  question  was  tested  by 
question  3  of  the  telephone  survey  (see  Appendix  A).  Out  of 
the  130  individuals  interviewed,  57%  believed  that  technical 
order  acquisition  planning  began  early  enough  to  be  suc¬ 
cessful.  When  the  responses  of  the  CTOM  members  were  removed 
from  the  data,  and  only  the  responses  of  the  four  divisions 
who  actively  worked  in  the  technical  order  acquisition 
process  were  examined,  61%  of  those  respondents  believed 
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that  planning  began  early  enough.  Based  dn  these  results  of 
the  survey,  it  could  be  concluded  that  technical  order  plan¬ 
ning  generally  began  early  enough  in  the  acquisition  cycle 
to  satisfy  a  majority  of  the  individuals  responsible  for  the 
accomplishment  of  technical  order  acquisition.  This  indi¬ 
cates  that  the  problem  associated  with  research  question 
number  1  was  not  considered  a  valid  problem  by  a  majority 
the  respondents . 

Validity  of  Recommended  Solutions.  The  solutions 
recommended  to  affect  earlier  planning  for  technical  order 
acquisition  included  s 

1.  Technical  Order  Management  Center  (Question  7). 

2.  Skeleton  Documents  (Question  8). 

3.  Technical  Orders  As  A  Product  (Question  9) 

% 

Since  the  majority  of  'the  respondents  did  not  believe 
that  earlier  planning  for  technical  order  acquisition  was 
required,  there  is  no  need  for  these  solutions.  However, 
each  of  these  solutions  was  also  associated  with  other  prob¬ 
lems  and  will  be  d.^jcussed  in  detail  later. 

Relative  Priority  of  the  Problem.  The  relative 
priority  of  the  problems  associated  with  each  research  ques¬ 
tion  were  determined  using  the  responses  to  question  number 
1  (Appendix  A)  from  the  telephone  survey.  Since  question 
number  1  was  an  open-ended  question,  the  authors  had  to 
evaluate  the  responses  and  categorize  them  in  order  to  as¬ 
sess  the  relative  priority  of  the  problems.  Some  of  the  re- 
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s ponses  identified  problems  that  fit  into  more  than  one  re¬ 
search  category.  For  example,  a  response  that  identified 
validation/verification  as  the  most  important  problem  with 
the  technical  order  acquisition  process  was  categorized  as  a 
training  problem,  a  coordination  and  communication  problem, 
and  a  manning  problem.  This  was  done  because  the  problem  of 
validation/verification  was  identified  as  being  caused  by 
those  three  problems  in  the  literature  review  of  this  re¬ 
port.  As  a  result  of  this  multiple  categorization,  the  per¬ 
centages  presented  as  the  relative  priorities  of  the  four 
research  questions  will  not  total  100%.  The  percentages  that 
will  be  presented  should  be  useful  in  determining  relative 
importance  of  each  problem  associated  with  the  research 
questions*. 

For  the  first  research  question,  8  out  of  130  (6%) 
indicated  that  the  need  for  earlier  planning  was  the  most 
critical  problem  with  the  technical  order  acquisition 
process.  This  low  percentage  supports  the  conclusion  made 
earlier  the  this  was  not  a  major  problem  of  the  technical 
order  acquisition  process.  — 

Re lative  Priority  of  Recommended  Solutions.  Since 
the  problem  associated  with  this  research  question  was  not 
found  to  be  a  major  problem  of  the  technical  order  acqui¬ 
sition  process,  the  relative  priority  of  the  solutions 
recommended  to  solve  that  problem  will  not  be  discussed 
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here.  They  will  be  discussed  in  conjunction  with  other  re¬ 
search  questions  when  applicable. 

Remarks.  The  large  difference  of  opinion  between 
the  CTOM  members  and  the  other  groups  must  be  discussed.  The 
CTOM  members  responded  10  to  1  (91%)  that  earlier  planning 
was  needed.  Only  39%  of  the  respondents  who  worked  directly 
within  the  technical  order  acquisition  process  believed  this 
was  a  problem.  This  large  difference  in  opinion  may  indicate 
a  coordination/communication  problem  between  the  SPOs  and 
the  policy  makers. 

In  addition^  the  solution  recommending  the  purchase  of 

i 

technical  orders  as  a  product  rather  them  as  part  of  data 

j 

was  not  associated  with  any  of  the  other  research  questions/ 

I 

but  was  highly  supported  by  the  respondents  (91%).  This 
recommended  1  solution  will  be  discussed  in  detail  later  in 

i 

this  chapter  and  further  discussion  will  be  made  in  the  next 

i 

chapter .  | 

| 

Research  Question  Number  2.  The  second  research  ques¬ 
tion  was: 

2.  How  can  communication  and  coordination  between 
technical  order  acquisition  agencies  and  other 
agencies  be  improved? 

Validity  of  the  Problem.  The  validity  of  the 
problem  associated  with  this  research  question  was  tested  by 
question  4  of  the  telephone  survey.  Out  of  the  130  individ¬ 
uals  interviewed/  73%  believed  that  coordination  and  com- 
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munication  between  technical  order  acquisition  agencies  and 
other  acquisition  agencies  was  a  valid  problem. 

Based  on  this  result  of  the  survey#  it  appeared  that 
coordination  and  communication  was  a  problem. 

Validity  of  Recommended  Solutions.  The  solutions 
recommended  to  solve  the  communication  and  coordination 
problem  included: 

1.  Establishment  of  a  technical  order  acquisition 
management  center. 

2.  Establishment  of  skeleton  Statement  Of  Needs  (SONs)# 
Requests  For  Proposals  (RFPs)#  and  contracts. 

3.  Establishment  of  a  handbook  identifying 
coordination/communication  responsibilities . 

The  validity  of  these  recommended  solutions  was  tested 
by  questions  7,  8#  and  10  respectively.  1 

Technical  Order  Management  Center.  The 
* 

recommendation  for  the  establishment  of  a  technical  order 
management  center  was  tested  by  question  number  7  ( see 
Appendix  A)  of  the  telephone  survey.  Out  of  the  130  respon¬ 
dents#  62%  believed  that  a  centralized  technical  order 
management  center  would  improve  the  technical  order  acqui¬ 
sition  process.  Based  on  this  percentage#  it  appeared  that 
the  establishment  of  a  centralized  technical  order  acqui¬ 
sition  management  center  was  a  valid  recommendation  to  im¬ 
prove  the  technical  order  acquisition  process. 

Skeleton  Documents .  The  recommendation  for 
the  establishment  of  skeleton  documents  was  tested  by  ques¬ 
tion  number  8  (see  Appendix  A)  of  the  telephone  survey.  Out 


of  the  130  respondents/  81%  believed  that  skeleton  documents 
would  improve  the  technical  order  acquisition  process.  Based 
on  this  percentage/  it  appeared  that  the  establishment  of 
skeleton  documents  was  a  valid  recommendation  to  improve  the 
technical  order  acquisition  process. 

Handbook.  The  recommendation  for  the  estab¬ 
lishment  of  a  handbook  to  help  identify  coordination  and 
communication  responsibilities  between  technical  order 
acquisition  agencies  and  other  system  acquisition  agencies 
was  tested  by  question  number  10  (see  Appendix  A)  of  the 
telephone  survey.  Out  of  the  130  respondents/  81%  believed 
that  a  handbook  on  coordination/communication  responsibili¬ 
ties  would  improve  the  technical  order  acquisition  process. 
Based  on  this  percentage,  it  appeared  that  the  establishment 
of  such  a  handbook  was  a  valid  recommendation  to  improve  the 
technical  order  acquisition  process. 

Relative  Priority  of  the  Problem.  The  responses 
to  question  number  1  were  used  to  determine  the  relative 
priority  of  this  problem.  Based  on  those  responses  (see 
Appendix  B) ,  93  out  of  130  (72%)  either  directly  or  indi¬ 
rectly  identified  coordination/communication  as  the  most 
significant  problem  with  the  technical  order  acquisition 
process.  This  result  indicated  that  the  respondents  believed 
this  to  be  a  major  problem  of  the  technical  order  acqui¬ 
sition  process. 


59 


Relative  Priority  of  Recommended  Solutions.  The 
responses  to  question  2  (see  Appendix  A  and  Appendix  C)  were 
used  to  determine  the  relative  priorities  of  the  solutions 
recommended  to  solve  this  problem. 

Centralized  Management  Center.  An  examin¬ 
ation  of  the  responses  to  question  number  2  (see  Appendix  C) 
resulted  in  the  identification  of  57  out  of  130  (44%)  that 
either  directly  or  indirectly  identified  the  establishment 
of  a  centralized  technical  order  management  center  as  the 
most  significant  change  that  could  be  made  to  the  technical 
order  acquisition  process. 

Skeleton  Documents .  An  examination  of  the 
responses  to  question  number  2  (see  Appendix  C)  resulted  in 
the  identification  of  24  out  of  130  (18%)  that  either  di¬ 
rectly  or  indirectly  identified  the  establishment  of  skele¬ 
ton  documents  as  the  most  significant  change  that  could  be 
made  to  technical  order  acquisition  process. 

Handbook.  An  examination  of  the  responses  to 
question  number  2  (see  Appendix  C)  resulted  in  the  identifi¬ 
cation  of  13  out  of  130  (10%)  that  either  directly  or  indi¬ 
rectly  identified  the  establishment  of  a  handbook  identi¬ 
fying  coordination/communication  responsibilities  as  the 
most  significant  change  that  could  be  made  to  the  technical 
order  acquisition  process. 

Remarks .  A  majority  of  the  respondents  indicated 
that  the  coordination  and  communication  between  technical 
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order  acquisition  agencies  and  other  agencies  needed  to  be 
improved.  The  respondents  supported  each  of  the  three  solu¬ 
tions  recommended  to  improve  that  coordination  and  communi¬ 
cation.  In  addition,  the  respondents  identified  the  estab¬ 
lishment  of  a  centralized  technical  order  management  center 
as  having  the  highest  priority  of  the  three  recommended 
solutions . 

Research  Question  Number  _3*  The  third  research  ques¬ 
tion  was : 

3.  How  can  manning  and  retention  of  technical  order 
acquisition  personnel  be  improved? 

Validity  of  the  Problem.  The  validity  of  the 
problem  associated  with  this  research  question  was  tested  by 
question  5  (see  Appendix  A)  of  the  telephone  survey.  Out  of 
the  130  individuals  interviewed,  74%  believed  that  the  man¬ 
power  dedicated  to  technical  order  acquisition  was  not  ade¬ 


quate. 

Based  on  this  result  of  the  survey,  it  appeared  that 
manning  was  a  valid  problem. 


Validity  of  Recommended  Solutions.  The  solutions 


recommended  to  solve  the  manning  and  retention  problem  , 
included : 

1.  Establishment  of  a  separate  AFSC  for  technical  order 
acquisition  managers.  \ 


2.  Establishment  of  a  technical  order  acquisition 
management  center. 

The  validity  of  these  recommended  solutions  was  tested 


by  questions  11  and  7  respectively. 
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Separate  AFSC.  The  recommendation  for  a  sep 
arate  AFSC  for  technical  order  acquisition  managers  was 
tested  by  question  number  11  (see  Appendix  A)  of  the  tele¬ 
phone  survey.  Out  of  the  130  respondents,  62%  believed  that 
some  form  of  career  field  or  AFSC  would  improve  the  tech¬ 
nical  order  acquisition  process.  Based  on  this  percentage, 
it  appeared  that  the  establishment  of  some  form  of  career 
field  for  technical  order  acquisition  managers  was  a  valid 
recommendation  to  improve  the  technical  order  acquisition 
process. 

Technical  Order  Management  Center.  The  rec¬ 
ommendation  for  the  establishment  of  a  technical  order 
management  center  was  tested  by  question  number  7  (see 
Appendix  A)  of  the  telephone  survey.  Out  of  the  130  respon¬ 
dents,  62%  believed  that  a  centralized  technical  order 
management  center  would  improve  the  technical  order  acqui¬ 
sition  process.  Based  on  this  percentage,  it  appeared  that 
the  establishment  of  a  centralized  technical  order  acqui¬ 
sition  management  center  was  a  valid  recommendation  to  im¬ 
prove  the  technical  order  acquisition  process. 

Relative  Priority  of  the  Problem.  The  responses 
to  question  number  1  were  used  to  determine  the  relative 
priority  of  this  problem.  Based  on  those  responses  (see 
Appendix  B),  17  out  of  130  (13%)  either  directly  or  indi¬ 
rectly  identified  manning  as  the  most  significant  problem 
with  the  technical  order  acquisition  process. 


Relative  Priority  of  Recommended  Solutions.  The 
responses  to  question  2  (see  Appendix  A  and  Appendix  C)  were 
used  to  determine  the  relative  priorities  of  the  solutions 
recor'unended  to  solve  this  problem. 

Separate  AFSC.  An  examination  of  the  re¬ 
sponses  to  question  number  2  (see  Appendix  C)  resulted  in 
the  identification  of  9  out  of  130  (7%)  that  either  directly 
or  indirectly  identified  the  establishment  of  a  separate 
career  field  or  AFSC  for  technical  order  acquisition  manag¬ 
ers  as  the  most  significant  change  that  could  be  made  to  the 
technical  order  acquisition  process. 

Centralized  Management  Center.  An  examin¬ 
ation  of  the  responses  to  question  number  2  (see  Appendix  C) 
resulted  in  the  identification  of  57  out  of  130  (44%)  that 
either  directly  or  indirectly  identified  the  establishment 
of  a  centralized  technical  order  management  center  as  the 
most  significant  change  that  could  be  made  to  the  technical 
order  acquisition  process. 

Remarks.  It  must  be  noted  that  the  term  "separate 
AFSC"  is  used  to  describe  a  separate  "career  field,"  which 
is  a  much  broader  topic  than  generally  associated  with  the 
term  "separate  AFSC";  The  two  terms  are  used  interchangeably 
to  indicate  a  method  of  providing  a  way  to  allow  career  pro¬ 
gression  and  promotion  within  the  technical  order  acqui¬ 
sition  management  field.  The  lack  of  potential  for  career 
progression  within  the  technical  order  acquisition  field 
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could  be  responsible  for  the  high  number  of  inexperienced 
personnel  working  in  the  field.  It  appeared  that  a  modifica¬ 
tion  to  the  structure  of  the  career  field  that  would  allow 
for  career  progression  was  much  preferred  over  establishing 
a  separate  AFSC  for  a  career  field  with  no  career  progres¬ 
sion  potential.  This  problem  will  be  discussed  further  in 
the  next  chapter. 

Research  Question  Number  A.  The  fourth  research  ques¬ 
tion  was* 

4.  How  can  training  and  knowledge  of  technical  order 
acquisition  personnel  be  improved? 

Validity  of  the  Problem.  The  validity  of  the 
problem  associated  with  this  research  question  was  tested  by 
question  6  (see  Appendix  A)  of  the  telephone  survey.  Out  of 
the  130  individuals  interviewed,  Q2%  believed  that  the 
training  and  knowledge  level  of  technical  order  acquisition 
personnel  was  not  adequate. 

Based  on  this  result  of  the  survey,  it  appeared  that 
training  was  a  valid  problem.  In  fact,  the  percentage  of  the 
respondents  that  believed  training  was  a  problem  was  the 
highest  percentage  associated  with  any  of  the  problems  in¬ 
vestigated  in  this  research. 

Vdiility  of  Recommended  Solutions.  The  solutions 
recommended  to  solve  the  training  problem  included: 

1.  Establishment  of  a  separate  AFSC  or  career  field  for 
technical  order  acquisition  personnel. 

2.  Establishment  of  a  centralized  technical  order 
acquisition  management  center. 


64 


3.  Establishment  of  a  handbook  on  coordination  and 
communication  responsibilities. 

The  validity  of  these  recommended  solutions  was  tested 
by  questions  11#  7#  and  10  respectively. 

Separate  AFSC.  The  recommendation  for  a 
separate  AFSC  for  technical  order  acquisition  managers  was 
tested  by  question  number  11  (see  Appendix  A)  of  the  tele¬ 
phone  survey.  Out  of  the  130  respondents#  62%  believed  that 
some  form  of  career  field  or  AFSC  would  improve  the  tech¬ 
nical  order  acquisition  process.  Based  on  this  percentage# 
it  appeared  that  the  establishment  of  some  form  of  career 
field  for  technical  order  acquisition  managers  was  a  valid 
recommendation  to  improve  the  technical  order  acquisition 
process. 

Technical  Order  Management  Center.  The  re¬ 
commendation  for  the  establishment  of  a  technical  order  man¬ 
agement  center  was  tested  by  question  number  7  (see  Appendix 
A)  of  the  telephone  survey.  Out  of  the  130  respondents,  62%  k 
believed  that  a  centralized  technical  order  management 
center  would  improve  the  technical  order  acquisition  pro¬ 
cess.  Based  on  this  percentage#  it  appeared  that  the  estab¬ 
lishment  of  a  centralized  technical  order  acquisition  man¬ 
agement  center  was  a  valid  recommendation  to  improve  the 
technical  order  acquisition  process. 

Handbook.  The  recommendation  for  the  estab¬ 
lishment  of  a  handbook  to  help  identify  coordination  and 


communication  responsibilities  between  technical  order 
acquisition  agencies  and  other  system  acquisition  agencies 
was  tested  by  question  number  10  (see  Appendix  A)  of  the 
telephone  survey.  Out  of  the  130  respondents ,  81%  believed 
that  a  handbook  on  coordination/communication  responsibil¬ 
ities  would  improve  the  technical  order  acquisition  process. 
Based  on  this  percentage ,  it  appeared  that  the  establishment 
of  such  a  handbook  was  a  valid  recommendation  to  improve  the 
technical  order  acquisition  process. 

Relative  Priority  of  the  Problem.  The  responses 
to  question  number  1  were  used  to  determine  the  relative 
priority  of  this  problem.  Based  on  those  responses  (see 
Appendix  B),  104  out  of  130  (80%)  either  directly  or  indi¬ 
rectly  identified  training  as  the  most  significant  problem 
with  the  technical  order  acquisition  process.  In  fact,  the 
percentage  of  respondents  (80%)  that  believed  training  was 
the  most  significant  problem  with  the  technical  order  acqui¬ 
sition  process  was  higher  than  the  percentages  associated 
with  any  of  the  other  problems  investigated  in  this 
research. 

Relative  Priority  of  Recommended  Solutions.  Al¬ 
though!  the  solutions  recommended  to  solve  the  training  prob¬ 
lem  have  already  been  discussed  in  conjunction  with  other 
problems,  they  will  be  restated  below. 

Centralized  Management  Center.  Out  of  the 
130  respondents,  44%  identified  the  establishment  of  a  cen- 
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tralized  technical  order  management  center  as  the  most  sig¬ 
nificant  change  that  could  be  made  to  the  technical  order 
acquisition  process.  / 

Handbook.  Out  of  the  130  respondents,  10% 
identified  the  establishment  of  a  handbook  on  coordination 
and  communication  responsibilities  as  the  most  significant 
change  that  could  be  made  to  the  technical  order  acquisition 
process. 

Separate  AFSC.  Only  7%  of  the  respondents 
identified  the  establishment  of  a  separate  career  field  or 
AFSC  for  technical  order  acquisition  managers  as  the  most 
significant  change  that  could  be  made  to  the  technical  order 
acquisition  process. 

Remarks .  Training  appeared  to  be  the  largest 
single  problem  with  the  technical  order  acquisition  process. 
In  fact,  it  was  very  difficult  to  separate  training  problems 
from  manning  problems  or  any  of  the  other  problems  inves¬ 
tigated.  Training  referred  to  much  more  than  formal  edu¬ 
cation.  It  also  included  experience  and  "corporate 
knowledge . " 

Other  Conclusions 

In  the  process  of  this  research,  new  and  unexpected 
information  was  discovered.  Some  of  that  unexpected  infor¬ 
mation  may  be  relevant  to  future  research.  Among  those  items 
that  may  be  relevant  to  future  research  was  the  recommen¬ 
dation  for  the  establishment  of  technical  orders  as  a  sep- 
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arate  product.  In  addition/  the  specific  responses  to  ques¬ 
tion  number  1  (see  Appendix  B)  and  question  number  2 
(see  Appendix  C)  presented  some  information  that  might  be 
relevant  to  future  research.  Finally/  the  problem  of  design¬ 
ing  one  technical  order  acquisition  management  process' that 
will  function  effectively  within  the  different  organization¬ 
al  structures  of  the  four  system  acquisition  divisions  in¬ 
vestigated  in  this  research  may  be  relevant  to  future 
research. 

Technical  Orders  as  a  Product.  The  recommendation  for 
the  establishment  of  technical  orders  as  a  separate  product 
in  contracts  was  associated  with  the  need  for  earlier  tech¬ 
nical  order  acquisition  planning.  Since  the  majority  of  the 
respondents  believed  that  earlier  technical  order  acqui¬ 
sition  planning  was  not  needed/  this  recommended  solution 
was  not  discussed  earlier.  However/  91%  of  the  respondents 
believed  that  this  recommended  solution  could  improve  the 
technical  order  acquisition  process.  Because  such  a  high 
percentage  of  the  respondents  supported  this  recommended 
solution/  it  will  be  discussed  here.  Question  9  (see  Appen¬ 
dix  A)  of  the  telephone  survey  was  used  to  measure  the  per¬ 
centage  of  the  respondents  who  supported  this  recommended 
solution. 

Relative  Priority  of  the  Solution.  An  examination 
of  the  responses  to  question  number  2  (see  Appendix  C)  re¬ 
sulted  in  the  identification  of  15  out  of  130  (12%)  that 
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either  directly  or  indirectly  identified  the  establishment 
of  technical  orders  as  a  separate  product  as  the  most  sig¬ 
nificant  change  that  could  be  made  to  the  technical  order 
acquisition  process. 

Remarks .  This  recommended  solution  had  the  high¬ 
est  support  (91%)  of  the  solutions  examined  in  this  re¬ 
search.  It  was  a  valid  solution  and  will  be  included  in  the 
final  recommendations  of  this  research. 

It  must  be  noted  that  the  Army  and  Navy  do  purchase 
technical  orders  as  a  separate  product/  and  the  Air  Force 
has  done  so  on  a  very  limited  basis. 

Responses  to  Question  1,.  Question  number  1  (see  Appen¬ 
dix  A  and  Appendix  B)  was  unstructured  and  was  designed  to 
find  any  problems  that  may  have  been  omitted  in  this  re¬ 
search.  Out  of  the  130  responses,  only  10  (8%)  did  not  fit 
into  one  or  more  of  the  problem  areas  investigated  in  this 
research.  It  appeared  that  the  problems  investigated  in  this 
research  were  the  major  problems  of  the  technical  order 
acquisition  process. 

As  was  mentioned  earlier,  some  responses  to  this  ques¬ 
tion  were  categorized  into  one  or  more  of  the  problems  in¬ 
vestigated  in  this  research.  Several  responses  appeared 
frequently  enough  to  warrant  individual  attention  here. 

Those  responses  are: 

1 .  Validation/verification . 

2.  Obtaining  technical  orders  at  the  same  time  that  the 
equipment  is  delivered. 


3.  Relative  priority  of  technical  orders  in  the  system 
acquisition  process. 

Validation/Verification.  As  was  mentioned  ear¬ 
lier#  problems  with  validation  and  verification  were  con¬ 
sidered  to  be  problems  of  coordination/communication#  man¬ 
ning#  and  training  for  the  purpose  of  this  research.  How¬ 
ever#  8  of  the  130  (6%)  respondents  identified  validation  or 
verification  as  the  most  significant  problem  with  the  tech¬ 
nical  order  acquisition  process.  This  percentage  was  sig¬ 
nificant  enough  to  suggest  that  solutions  aimed  at  improving 
coordination  and  communication#  manning#  and  training  should 
specifically  address  validation/verification  problems. 

Obtaining  Technical  Orders  on  Time.  The  problem 
of  obtaining  technical  orders  at  the  same  time  that  the 
equipment  is  delivered  was  considered  a  coordination  and 
communication,  and  a  training  problem  for  the  purpose  of 
this  research.  However#  12  of  the  130  (9%)  respondents  iden¬ 
tified  this  as  the  most  significant  problem  with  the  tech¬ 
nical  order  acquisition  process.  This  percentage  was  signif¬ 
icant  enough  to  suggest  that  solutions  aimed  at  improving 
coordination/communication  and  training  should  specifically 
address  the  problem  of  obtaining  technical  orders  at  the 
same  time  that  the  equipment  is  delivered. 

Priority  of  Technical  Order  Acquisition.  The 
problem  of  the  relatively  low  priority  of  technical  order 
acquisition  in  the  system  acquisition  process  was  considered 
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a  coordination/communication  and  training  problem  for  the 
purpose  of  this  research.  However,  18  out  of  130  (14%)  re¬ 
spondents  identified  the  relatively  low  priority  of  techni¬ 
cal  order  acquisition  needs  within  the  system  acquisition 
process  as  the  most  significant  problem  of  the  technical 
order  acquisition  process.  This  percentage  was  significant 
enough  to  suggest  that  solutions  aimed  at  improving  coordi¬ 
nation/communication  and  training  specifically  address  the 
problem  of  the  priority  of  technical  order  acquisition  in 
the  system  acquisition  process. 

Establishing  technical  orders  as  a  separate  product 
might  help  to  raise  the  priority  of  technical  orders  in  the 
system  acquisition  process  since  technical  orders  will  be- 
come  a  contractual  responsibility. 

Responses  to  Question  2.  Question  number  2  (see  Appen¬ 
dix  A  and  Appendix  C)  was  an  unstructured  question  designed 
to  identify  any  solutions  that  may  have  .  een  omitted  in  this 
research.  Of  the  130  responses  to  this  question,  only  21 
(16%)  did  not  fit  into  one  or  more  of  the  recommended  solu¬ 
tions  identified  in  this  research.  Of  this  21,  over  half 
(11)  pertained  to  automating  technical  orders.  The  remaining 
10  responses  were  isolated  and  not  related. 

The  topic  of  automated  technical  orders  is  in  itself 
worthy  of  a  research  project.  For  the  purpose  of  this  re¬ 
search,  automated  technical  orders  were  considered  as  a 
format  for  technical  orders  and  were  considered  to  be  out- 
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side  of  the  scope  of  this  investigation  into  the  problems  of 
the  technical  order  acquisition  process.  It  must  be  noted, 
however,  that  it  is  very  likely  that  the  technical  order 
acquisition  process  will  have  to  acquire  automated  technical 
orders  in  the  future,  and  any  changes  to  the  technical  order 
acquisition  process  recommended  in  this  research  will  have 
to  accommodate  that  possibility.  This  topic  will  be  dis¬ 
cussed  in  more  detail  in  the  next  chapter. 

Problems  Resulting  from  Organizational  Differences.  As 
was  mentioned  earlier,  the  four  Divisions  that  were  investi¬ 
gated  differed  in  their  organizational  structure  and  in 
their  technical  order  acquisition  processes.  This  diversity 
had  to  be  considered  when  investigating  the  problems  of  the 
technical  order  acquisition  process.  Whatever  solutions  are 
recommended  as  a  result  of  this  research  must  be  effective 
in  all  of  the  divisions,  not  just  one. 

Since  the  majority  of  the  divisions  (BMO,ESD,  and  AD) 
are  matrix  type  organizations,  and  generally  do  not  have 
technical  order  acquisition  personnel  assigned  on  a  one-for- 
one  basis  to  a  system,  the  technical  order  acquisition  pro¬ 
cess  must  be  able  to  accommodate  this  type  of  organization. 

The  recommendations  that  will  be  presented  in  the 
following  chapter  will  attempt  to  accommodate  the  matrix 
organization  as  well  as  the  organization  with  technical 
order  acquisition  personnel- assigned  on  a  one-for-one  basis 
with  a  system. 
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Chapter  Summary 

The  intent  of  this  research  was  to  validate  four  re¬ 
search  questions  and  the  solutions  recommended  to  solve 
those  problems.  Three  of  the  four  research  questions  were 
found  to  be  valid.  The  need  for  technical  order  acquisition 
considerations  to  begin  earlier  in  the  system  acquisition 
process  was  not  found  to  be  valid.  All  recommended  solutions 
were  found  to  be  valid. 

Problems .  The  responses  to  the  problems  associated 
with  the  research  questions  were  as  follows: 

1.  Are  training  and  knowledge  of  technical  order 
acquisition  personnel  adequate?  (82%  said  no). 

2.  Is  manpower  dedicated  to  technical  order  acquisition 
adequate?  (74%  said  no). 

3.  Are  coordination  and  communication  between  technical 
order  acquisition  agencies  and  other  agencies 
adequate?  (73%  said  no). 

4.  Does  planning  for  technical  order  acquisition  begin 
early  enough?  (43%  said  no). 

The  relative  priorities  of  these  problems  were  also 
discussed.  The  training  problem  was  given  the  highest  (80%) 
priority,  while  the  coordination/communication  problem  was 
given  the  second  highest  (72%)  priority. 

Recommended  Solutions.  Responses  to  recommended  solu¬ 
tions  were  as  follows: 

1.  Do  technical  orders  need  to  be  treated  as  a  separate 
line  item  product?  (91%  said  yes). 

2.  Is  a  handbook  identifying  coordination/communication 
responsibilities  needed?  (81%  said  yes). 


3.  Are  skeleton  documents  needed?  (81%  said  yes). 

4.  Is  a  centralized  technical  order  management  center 
needed?  (62%  said  yes). 

5.  Is  a  technical  order  acquisition  management  AFSC  or 
career  field  needed?  (62%  said  yes). 

Relative  priorities  for  the  recommended  solutions  were 
also  given.  The  need  for  a  centralized  technical  order 
management  agency  was  given  the  highest  (44%)  priority. 

Final  Conclusions .  This  research  identified  three 
major  problems  with  the  technical  order  acquisition  process 
and  identified  five  valid  solutions  to  solve  those  problems. 
This  information  should  give  decision  makers  a  starting 
point  in  their  efforts  to  improve  the  technical  order  acqui¬ 
sition  process.  The  next  step  is  to  determine  the  best  way 
to ’implement  these  recommended  solutions.  This  will  require 
further  investigation;  however,  some  recommendations  will  be 
made  in  the  following  chapter. 
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VI.  Recommendations 


The  objective  of  this  research  was  to  identify  problems 
with  the  acquisition  of  U.S.  Air  Force  technical  orders,  and 
identify  possible  changes  to  the  technical  order  acquisition 
process  that  would  solve  those  problems.  That  objective  has 
been  accomplished. 

The  five  recommended  solutions  investigated  in  this 
research  were  found  to  be  valid  and  should  be  implemented 
into  the  technical  order  acquisition  process. 

The  following  recommendations  are  offered  as  a  starting 
point  for  the  further  research  that  will  be  required  to 
develop  an  integrated  plan  for  implementing  those  recom¬ 
mended  solutions.  Recommendations  for  research  into  new 
areas  discovered  during  the  course  of  this  research  will 
also  be  presented. 

Integrated  Implementation  Plan 

The  results  obtained  in  this  research  were  based  on  the 
technical  order  acquisition  process  as  it  existed  at  the 
time  this  report  was  written.  Once  any  of  the  solutions 
identified  in  this  research  are  implemented,  the  technical 
order  acquisition  process  will  be  changed.  As  a  result,  the 
implementation  of  one  recommended  solution  may  affect  the 
need  for  another  recommended  solution. 

In  order  to  insure  that  the  most  effective  plan  is 
recommended,  the  importance  of  each  recommended  solution 
must  be  determined.  The  importance  of  the  five  solutions 


recommended  in  this  research  was  determined  by  the  relative 
priority  of  each  solution  described  in  the  preceding  chap¬ 
ter.  Those  relative  priorities  were  as  follows: 

1.  Establishment  of  a  technical  order  acquisition 
management  center  ( 44%) . 

2.  Establishment  of  skeleton  documents  (18%). 

3.  Establishment  of  technical  orders  as  a  separate 
product  (12%). 

4.  Establishment  of  a  handbook  identifying 
coordination/communications  requirements  (10%). 

5.  Establishment  of  a  separate  AFSC/career  field  for 
technical  order  managers  (7%). 

The  percentages  listed  above  represent  the  respondents 

who  indicated  that  the  associated  solution  was  the  single 

most  important  change  that  could  be  made  to  the  technical 

order  acquisition  process. 

The  implementation  of  the  above  solutions  will  be 
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discussed  in  relation  to  the  four  problems  investigated  in 
this  research.  The  importance  of  the  four  problems  was 
determined  by  the  relative  priority  of  each  problem 

I  # 

described  in  the  preceding  chapter.  Those  relative 
priorities  were  as  follows: 

1.  Need  for  better  training  (80%). 

2.  Need  for  better  coordination/communication  (72%). 

3.  Need  for  better  manning  for  technical  order 
acquisition  (13%) . 

4.  Need  for  earlier  planning  for  technical  order 
acquisition  (6%). 

The  percentages  listed  above  represent  the  respondents 
who  indicated  that  the  associated  problem  was  the  single 
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most  significant  problem  vith  the  technical  order  acquis¬ 
ition  process.  The  percentages  add  up  to  more  than  10056 
because  of  the  open-ended  nature  of  the  question  used  to 
determine  the  percentages  and  the  assignment  of  some  re¬ 
sponses  into  more  than  one  problem  area. 

The  need  for  earlier  planning  will  be  discussed  in 
these  recommendations  even  though  the  majority  of  the 
respondents  indicated  that  it  was  not  a  problem.  It  will  be 
included  for  two  reasons: 

1.  The  solutions  associated  with  this  problem  were 
also  associated  with  other  problems  that  were  found 
to  be  significant.  The  implementation  of  those 
solutions  in  conjunction  with  those  other  problems 
will  make  it  possible  for  earlier  planning  of 
technical  order  acquisition  with  little  or  no 
additional  cost. 

2.  Even  though  a  majority  of  the  respondents  did  not 
consider  this  a  problem,  43%  did  indicate  that  it 
was  a  problem.  • 

As  a  result,  it  seems  reasonable  to  include  this 
problem  in  the  following  recommendations. 

Centralized  Management  Center.  The  recommendation  to 
establish  a  centralized  technical  order  acquisition 
management  center  attained  the  highest  relative  priority 
rating  (44%)  of  the  five  recommended  solutions.  This 
recommended  solution  was  also  associated  with  all  four 
problems  identified  in  this  research.  As  a  result,  this 
recommended  solution  will  form  the  nucleus  of  the  inte¬ 
grated  implementation  plan  that  will  be  recommended. 

Since  this  research  began,  the  Air  Force  has  taken 
steps  to  establish  a  centralized  technical  order  management 
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agency.  Initial  plans  call  for  an  Air  Force  Technical  Order 
Center  (AFTOC)  to  be  established  at  the  Air  Force 
Acquisition  Logistics  Center  (AFALC)  at  Wright-Patterson 
AFB,  Ohio.  The  final  plan  for  the  AFTOC  is  expected  in  1985. 

The  following  recommendations  should  be  implemented 
through  a  gradual  evolutionary  process  and  not  as  a  rapid 
change  to  the  system. ‘The  framework  should  be  established 
and  then  be  allowed  to  grow  only  after  needs  and  procedures 
have  been  validated  on  a  small  scale. 

In  order  to  maximize  the  impact  of  the  Air  Force 
Technical  Order  Center  on  the  four  problems  identified  in 
this  research,  three  divisions  of  responsibility  should  be 
established.  Those  three  divisions  should  be  field  assis¬ 
tance,  technical  specialization,  and  plans  and  programs.  The 
term  "division"  will  be  used  in  this  discussion  to  identify 
that  portion  of  the  AFTOC  that  would  be  assigned  each  of  the 
three  divisions  of  responsibility  mentioned  above.  The  size 
of  those  "divisions"  and  their  actual  titles  should  be 
determined  as  part  of  the  final  implementation  plan  for  the 
AFTOC. 

The  field  assistance  division  would  have  the  most 
direct  impact  on  the  four  problems  identified  in  this 
research?  however,  without  the  assistance  of  the  other  two 
divisions,  the  field  assistance  division  would  not  function 
effectively. 

Field  Assistance  Division.  The  field  assistance 
division  should  consist  of  experienced  technical  order 


78 


/ 


acquisition  personnel  who  should  be  assigned  the  respons¬ 
ibility  of  monitoring  specific  programs.  Individuals  from 
this  division  should  be  familiar  with  the  progress  of  their 
assigned  programs  and  serve  as  the  centralized  point  of 
contact  for  any  technical  order  related  advice  required  by 
the  System  Program  Office. 

This  division  could  have  an  impact  on  all  four  of  the 
problems  identified  in  this  research. 

Training.  The  field  assistance  division 
could  have  a  significant  impact  on  the  training  problem.  By 
monitoring  the  technical  order  acquisition  progress  of 
specific  programs  and  being  an  advisor  for  those  programs* 
this  division  could  introduce  additional  expertise  and 
knowledge  into  the  technical  order  acquisition  process.  This 
additional  expertise  and  knowledge  would  be  available  co  all 
SPOs  irregardless  of  their  organizational  structure.  The 
field  assistance  personnel  of  the  AFTOC  would  be  able  to 
assist  PMs,  DPMLs,  or  TOMAs  equally  as  effectively. 

When  the  field  assistance  division  is  first  estab¬ 
lished,  personnel  should  be  assigned  to  monitor  the  programs 
that  have  the  least  experienced  technical  order  acquisition 
personnel  assigned  to  them.  This  would  probably  include 
programs  with  small  ILS  staffs  due  to  the  nature  of  the 
organization  or  the  stage  of  development  of  the  program. 

This  division  should  gradually  expand  to  cover  as  many 
programs  as  can  effectively  be  handled.  In  addition,  this 
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division  should  be  to  be  available  to  assist  any  program 
with  specific  problems. 

It  must  be  pointed  out  that  the  personnel  of  this 
division  should  not  be  alone  in  their  responsibilities.  They 
should  use  the  resources  of  the  other  AFTOC  divisions  in  the 
accomplishment  of  their  duties.  The  resources  of  the  tech¬ 
nical  specialist  division  will  be  extremely  important  to  the 
success  of  this  division. 


Coordination/Communication.  The  field  assis¬ 
tance  division  should  certainly  have  an  impact  on  the 
coordination/communication  problem.  The  primary  respon¬ 
sibility  of  this  division  would  be  to  assist  the  SPOs.  One 
major  objective  of  that  assistance  should  be  to  insure  that 
proper  coordination  and  communication  is  accomplished  in  the 

a 

technical  order  acquisition  process  for  each  program. 

Manning .  This  division  should  also  have  an 
impact  on  the  manning  problem.  Since  the  personnel  of  this 
division  can  assist  SPO  personnel  with  technical  order 
acquisition  problems,  it  is  possible  that  the  need  for 
additional  technical  order  acquisition  personnel  in  the  SPOs 
will  be  reduced.  The  establishment  of  the  AFTOC  will  require 
an  addition  of  personnel  to  the  staff  level  of  the  technical 
order  acquisition  process.  It  is  possible  that  the  neeii  to 


increase  personnel  for  this  function  might  have  a  higher 
priority  than  the  need  to  increase  personnel  for  technical 
order  acquisition  on  specific  programs.  As  a  result,  it 
appears  that  additional  personnel  might  be  obtained  for  this 
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staff  function.  The  net  result  should  be  an  overall  increase 
in  manpower  for  the  technical  order  acquisition  process. 

Earlier  Planning.  The  field  assistance 
division  should  make  it  possible  for  technical  order  acqui¬ 
sition  planning  to  begin  earlier  on  some  programs.  If  field 
assistance  personnel  would  be  assigned  to  monitor  a  program 
as  soon  as  the  program  is  established,  then  technical  order 
relevant  data  could  be  compiled  from  the  beginning  of  the 
program.  Even  though  there  are  not  many  technical  order 
acquisition  related  activities  during  the  early  stages  of  a 
program,  the  field  assistance  division  would  be  available 
and  familiar  with  the  program  if  their  assistance  is  needed. 

The  Navy  performs  a  similar  function  at  its  Naval  Air 
Technical  Service  Facility.  The  Navy  system  might  be  a 
useful  reference  for  the  implementation  of  this  division  of 
the  AFTOC. 

Technical  Specialization  Division.  The  technical 
specialization  division  should  consist  of  personnel  who  are 
experts  on  specific  technical  aspects  of  technical  order 
acquisition.  Technical  specialists  should  be  responsible  for 
the  management  of  Air  Force  technical  order  specifications 
and  they  should  \  ^rticipate  in  the  DOD  Technical  Manual 
Specifications  and  Standards  (TMSS)  program.  Technical 
specialists  should  also  be  responsible  for  evaluating  and 
tracking  technical  order  cost  data.  These  are  a  few  possible 
areas  of  specialization  that  could  be  included  in  this 
division.  Further  research  is  required  to  identify  the 
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specific  areas  of  specialization  that  should  be  included  in 
this  division. 

The  personnel  of  this  division  should  serve  a  dual 
function.  They  should  assist  the  field  assistance  division 
with  the  technical  aspects  of  technical  order  acquisition. 
They  should  also  assist  the  plans  and  programs  division  in 
making  recommendations  for  improvements  to  technical  areas 
of  the  technical  order  acquisition  process. 

The  technical  specialization  division  should  impact  the 
four  problems  identified  in  this  research  through  the  assis¬ 
tance  that  it  gives  to  the  other  two  divisions  of  the  AFTOC. 
The  technical  specialists  should  add  "corporate  knowledge" 
to  the  technical  order  acquisition  system.  That  "corporate 
knowledge"  should  reach  the  SPOs  through  the  field  assis¬ 
tance  personnel  and  should  influence  Air  Force  policy 
through  the  plans  and  program  personnel  of  the  AFTOC. 

Plans  and  Programs  Division.  The  plans  and 
programs  division  should  have  three  major  responsibilities. 
This  division  should  have  an  input  into  policy  changes  and 
additions  that  affect  technical  order  acquisition.  It  should 
continually  evaluate  the  technical  order  acquisition  process 
and  identify  problems  and  weaknesses.  It  should  also  search 
for  new  ideas  and  technology  that  might  improve  technical 
order  acquisition. 

Inputs  to  Policy.  The  plans  and  programs 
division  should  have  personnel  assigned  to  attend  meetings 
and  be  on  committies  that  are  concerned  with  technical  order 


acquisition  related  policy.  This  includes  working  with  other 
services  on  Joint  Service  plans  and  programs  for  technical 
order  acquisition. 

Inputs  from  the  field  assistance  and  technical  special¬ 
ization  divisions  should  enable  personnel  of  this  division 
to  make  accurate  inputs  to  technical  order  acquisition 
policy  at  all  levels.  The  CTOM  Executive  Committee  investi¬ 
gated  in  this  report  is  an  example  of  the  type  of  committee 
that  plans  and  programs  division  personnel  should  be  in¬ 
volved  with.  Further  research  is  required  to  identify  the 
specific  policy  making  organizations  that  could  benefit  from 
inputs  of  the  plans  and  programs  division  of  the  AFTOC. 

Monitor  Process .  The  plans  and  programs 
division  of  the  AFTOC  should  have  personnel  assigned  to 

I 

monitor  the  technical  order  acquisition  process  and  to 
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investigate  any  potential  problems  as  they  are  discovered. 
This  responsibility  will  require  a  close  relationship  with 
the  field  assistance  division  and  the  technical  special¬ 
ization  division  since  those  divisions  should  be  the  first 
to  discover  any  problems. 

Once  possible  problems  are  identified ,  the  plans  and 
programs  personnel  should  investigate  them  and  recommend 
solutions  when  applicable.  They  should  also  recommend 
implementation  plans  for  recommended  solutions  when  re¬ 
quired.  Implementation  of  a  recommended  solution  may  require 
only  a  minor  change  to  the  system,  or  it  may  require  a 
policy  change.  In  the  case  of  a  minor  change,  this  division 
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should  be  able  to  effect  the  change  through  the  proper 
channels.  In  the  case  of  a  major  change  that  requires  a 
policy  change,  the  proper  information  should  be  given  to  the 
personnel  within  the  division  who  are  responsible  for  making 
technical  order  policy  recommendations. 

Even  though  many  problems  will  probably  be  identified 
through  the  resources  within  the  AFTOC,  other  organizations 
are  involved  with  technical  order  acquisition  and  should  not 
be  overlooked.  A  formal  pl?n  should  be  established  to  effect 
a  liaison  with  major  contractors  and  each  major  command  to 
get  their  inputs  to  technical  order  acquisition  related 
problems. 

New  Ideas  and  Technology.  The  clans  and 
programs  division  of  the  AFTOC  should  also  hhve  personnel 
assigned  to  examine  new  ideas  and  technology  that  might 
improve  the  technical  order  acquisition  process.  Computer¬ 
ization  and  automation  could  significantly  affect  technical 
orders  and  the  acquisition  process  in  the  future.  Plans  and 
programs  personnel  from  the  AFTOC  should  be  involved  in  the 
development  of  any  new  technology  or  system  acquisition 
procedures  that  relate  to  technical  order  acquisition.  They 
should  also  develop  plans  to  ensure  that  the  technical  order 
acquisition  process  will  function  effectively  once  those  new 
technologies  or  changes  to  the  system  acquisition  process 
are  adopted. 

Additional  Comments  on  AFTOC.  The  establishment 
of  the  Air  Force  Technical  Order  Center  (AFTOC)  gives  the 


Air  Force  the  opportunity  to  make  improvements  to  the 
technical  order  acquisition  process  that  could  help  solve  or 
reduce  the  impact  of  all  four  problems  identified  in  this 
research.  A  thoroughly  researched  and  well  integrated 
implementation  plan  will  be  required  in  order  to  realize  the 
full  potential  of  that  op port unity. The  establishment  of  the 
AFTOC  will  be  a  major  change  to  the  technical  order  acqui¬ 
sition  process  and  will  require  changes  to  many  regulations 
and  procedures. 

The  changes  to  regulations  and  procedures  that  will 
result  from  the  establishment  of  the  AFTOC  will  effect  many 
AFSC  and  AFLC  organizations  that  are  involved  in  the  tech¬ 
nical  order  acquisition  process.  Those  organizations  should 
be  involved  in  the  planning  process  for  those  changes.  The 
result  of  that  planning  should  be  an  AFTOC  that  will  provide 
expertise  and  assistance  to  the  SPOs  and  technical  order 
policy  makers  from  a  centralized  location.  The  control  of 
policy  making  and  technical  order  acquisition  should  remain 
the  responsibility  of  AFSC  and  AFLC. 

The  recommendations  presented  above  identify  some 
responsibilities  that  should  be  investigated  in  conjunction 
with  the  implementation  plan  for  the  AFTOC.  They  are  based 
on  the  results  of  this  research?  however,  this  research  is 
very  limited  in  scope.  As  a  result,  these  recommendations 
only  address  AFTOC  responsibilities  for  technical  order 
acquisition  accomplished  through  a  System  Program  Office. 


Further  research  is  required  to  determine  AFTOC  respons¬ 
ibilities  for  other  technical  order  acquisitions. 

Skeleton  Documents .  The  recommendation  for  the  estab¬ 
lishment  of  skeleton  documents  obtained  a  relative  priority 
rating  of  18%.  This  recommended  solution  vas  associated  with 
the  coordination/communication  problem  and  the  need  for 
earlier  technical  order  acquisition  planning. 

The  implementation  plan  for  this  recommendation  will  be 
effected  by  the  implementation  plan  selected  for  the  AFTOC. 
The  purpose  of  skeleton  documents  would  be  to  assist  SPO 
personnel  with  technical  order  related  documentation .  if  the 
AFTOC  is  organized  as  outlined  in  the  preceding  recom¬ 
mendation  ,  skeleton  documents  may  not  be  needed.  AFTOC 
personnel  might  be  able  to  provide  better  assistance  to  the 
SPOs  than  could  be  provided  by  skeleton  documents.  Further 
research  is  definitely  required  before  any  implementation  of 
skeleton  documents  should  be  attempted. 

It  is  possible  that  skeleton  documents  may  be  found 
useful  even  if  the  AFTOC  does  perform  the  assistance  func¬ 
tion  recommended  earlier.  Skeleton  documents  could  still  be 
of  use  to  AFTOC  personnel  and  SPO  personnel  as  a  ^orm  of 
checklist . 

Further  research  into  the  implementation  plan  for  this 
recommendation  should  include  an  evaluation  of  the! skeleton 
documents  available  through  the  ESD  Computer  Generated 
Acquisition  Documents  System  ( CGADS ) .  Information  on  the 
CGADS  can  be  obtained  from  ESD/OCHE  at  Hanscom  AFB. 
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The  need  for  skeleton  documents  and  the  specific 
documents  that  should  be  skeletonized  should  be  determined 
only  after  the  exact  structure  of  the  AFTOC  has  been  de¬ 
termined.  The  final  determination  of  the  need  for  skeleton 
documents  and  the  plan  for  the  implementation  of  those 
documents  should  be  a  project  of  the  plans  and  programs 
division  of  the  AFTOC.  Inputs  from  the  field  assistance  and 
technical  specialization  divisions  should  be  used  to  build 
the  plan.  The  skeleton  documents  (if  required)  should  be 
built  by  the  technical  specialists  and  their  use  introduced 
to  the  SPOs  by  the  field  assistance  personnel.  The  plans  and 
programs  division  should  insure  that  any  changes  to  policy 
or  procedures  that  may  be  required  by  the  introduction  of 
skeleton  documents  to  the  technical  order  acquisition 
process  are  referred  to  the  proper  policy  makers. 

Technical  Orders  as  a.  Separate  Product .  The  recom¬ 
mendation  for  the  establishment  of  technical  orders  as  a 
separate  line  item  product  obtained  a  relative  priority  off 
12%.  This  recommended  solution  was  supported  by  91%  of  the 
respondents . 

Part  of  the  coordination  and  communication  problem 
might  have  been  a  result  of  the  low  priority  assigned  to 
technical  order  acquisition  by  contractors  and  SPO  person¬ 
nel.  Making  technical  orders  a  separate  contractual  obli¬ 
gation  might  help  raise  the  level  of  priority  assigned  to 
technical  order  acquisition  items. 


The  implementation  plan  for  this  recommended  solution 
should  be  developed  by  the  AFTOC.  The  plans  and  programs 
division  should  develop  the  plan  through  coordination  with 
the  technical  specialists  and  field  assistance  personnel. 

The  plan  should  be  approved  and  made  policy  through  the 
appropriate  channels. 

The  implementation  of  this  plan  will  require  changes  to 
technical  order  related  specifications  and  other  documents. 
The  technical  specialization  division  of  the  AFTOC  should  be 
involved  in  making  those  changes. 

Handbook.  The  recommendation  for  the  development  of  a 
handbook  outlining  coordination  and  communication  respon¬ 
sibilities  obtained  a  relative  priority  of  10%.  This  recom¬ 
mended  solution  was  associated  with  the  coordination  and 
communication  problem  and  the  training  problem. 

The  implementation  plan  for  this  recommendation  will 
also  be  effected  by  the  implementation  plan  selected  for  the 
AFTOC.  The  AFTOC  could  have  a  significant  impact  on  the 
coordination  and  communication  problem  as  well  as  the 
training  problem.  The  plans  and  programs  division  of  the 
AFTOC  should  investigate  this  recommended  solution.  That 
investigation  will  require  inputs  from  the  field  assistance 
division  and  the  technical  specialization  division  of  the 
AFTOC  as  well  as  other  technical  order  acquisition  organ¬ 
izations.  If  it  is  determined  that  a  handbook  is  still 
necessary  after  the  establishment  of  the  AFTOC,  the  plans 


88 


and  programs  division  should  be  responsible  for  the  develop¬ 
ment  of  the  handbook. 

Separate  AFSC.  The  recommendation  for  a  separate  AFSC 
or  career  field  for  technical  order  acquisition  personnel 
obtained  the  lowest  relative  priority  of  the  five  recom¬ 
mended  solutions.  This  recommended  solution  was  associated 
with  the  manning  and  training  problems  identified  in  this 
research. 

The  implementation  plan  for  this  recommended  solution 
will  be  affected  by  the  implementation  plan  selected  for  the 
AFTOC.  The  AFTOC  could  provide  new  opportunities  for  career 
advancement  within  the  technical  order  acquisition  field. 

So,  in  effect#  the  establishment  of  the  AFTOC  could  result 
in  a  viable  career  field  for  technical  order  acquisition 
personnel . 

As  indicated  by  the  small  number  of  technical  order 
acquisition  specialists  interviewed  in  this  research#  there 
were  not  enough  technical  order  acquisition  specialists  to 
justify  a  separate  AFSC.  However#  the  career  progression 
opportunities  that  could  be  made  possible  by  the  AFTOC  could 
in  effect  implement  the  career  field  aspect  of  this  recom¬ 
mended  solution. 

Final  Comments  on  Implementation  Plan.  The  establish¬ 
ment  of  the  AFTOC  is  the  key  to  the  integrated  implemen¬ 
tation  plan  recommended  above.  The  implementation  plans  or 
the  other  recommended  solutions  will  be  affected  by  the 
final  implementation  plan  selected  for  the  AFTOC.  The 

89 


/ 


resources  of  the  AFTOC  could  be  used  to  develop  the  imple¬ 
mentation  plans  for  those  other  recommended  solutions. 

Other  Recommendations 

Four  technical  order  acquisition  related  topics  should 
be  included  in  the  implementation  planning  for  the  solutions 
recommended  above.  Those  four  topics  are: 

1.  Technology  advances. 

2.  Validation/verification. 

3.  Insuring  that  technical  orders  arrive  in  the  field 
with  the  equipment. 

4.  Relative  priority  of  technical  orders  in  the  system 
acquisition  process. 

Technology  Advances .  Technological  advances  could  have 
a  significant  impact  on  technical  order  acquisition  in  the 
future.  Automation  of  technical  orders,  technological 
sophistication  of  future  weapon  systems,  and  technological 
improvements  to  equipment  used  in  the  system  acquisition 
process  could  all  have  an  impact  on  the  technical  order 
acquisition  process.  Automated  technical  orders  are  a 
technological  reality.  Technological  sophistication  of 
future  weapon  systems  could  effect  the  maintenance  and 
technical  order  support  required  for  those  systems.  Techno¬ 
logical  advances  in  equipment  used  for  development,  plan¬ 
ning,  and  control  of  the  system  acquisition  process  could 
effect  future  technical  order  acquisition  requirements. 

The  technical  order  acquisition  process  should  be 
prepared  for  any  responsibility  or  procedure  changes  that 
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might  result  from  technological  advances.  In  order  to  be 
prepared  for  those  changes ,  technical  order  acquisition 
personnel  should  be  involved  in  the  development  of  any 
technological  advances  that  could  affect  the  technical  order 
acquisition  process. 

Personnel  from  the  plans  and  programs  division  of  the 
AFTOC  should  monitor  research  efforts  that  might  result  in 
technological  advances.  When  technological  advances  are 
discovered  that  could  have  an  impact  on  the  technical  order 
acquisition  process/  plans  should  be  developed  to  prepare 
the  technical  order  acquisition  process  for  the  changes  that 
might  result  from  those  advances. 

Automated  technical  orders  sure  receiving  considerable 
attention  in  the  Air  Force  today.  The  technology  for  auto¬ 
mated  technical  orders  is  available  and  being  used  by 
non-defense  companies.  Defining  the  future  role  of  automated 
technical  orders  in  the  Air  Force  will  be  a  complex  task. 
That  task  should  be  accomplished  through  a  detailed  invest¬ 
igation  of  thu  need  for  automated  technical  orders. 
Researchers  should  be  careful  not  to  automatically  assume 
that  automated  technical  orders  are  needed  just  because  the 
technology  is  available.  The  assessment  of  the  need  for 
automated  technical  orders  should  be  based  on  the  projected 
improvement  to  operational  readiness  or  performance  that  can 
be  expected,  or  on  the  projected  cost  savings  that  can  be 
expected  without  reducing  performance  or  readiness.  This 
assessment  should  include  a  careful  examination  of  wartime 
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requirements  for  technical  orders.  Automated  technical 
orders  will  have  reliability  and  maintainability  costs  and 
considerations  associated  with  them  that  should  be  carefully 
evaluated. 

Validation/Verification.  The  implementation  plan  for 
the  solutions  recommended  in  this  research  should  address 
validation  and  verification  problems.  Two  major  problems 
associated  with  validation  and  varification  were  the  amount 
of  time  available  to  accomplish  those  tasks,  and  the  avail¬ 
ability  of  the  weapon  system  during  the  technical  order 
validation  and  verification  processes. 

The  establishment  of  technical  orders  as  a  separate 
product  could  have  am  impact  on  botl.  validation/verification 
problems.  If  the  contract  specifically  states  that  technical 
order  validation  and  verification  will  be  completed  by  a 
certain  time,  more  of  an  effort  might  be  made  towards 
meeting  the  contractual  responsibility.  A  penalty  for  not 
meeting  that  time  limit  might  provide  additional  incentive 
to  complete  validation  and  verification  on  time. 

The  AFTOC  field  assistance  division  could  help  the  SPOs 
insure  that  proper  attention  is  given  to  validation  and 
verification  considerations  throughout  the  system  acqui¬ 
sition  process.  Another  division  of  the  AFTOC,  the  technical 
specialization  division,  could  help  solve  these  problems  by 
providing  specialized  assistance  to  the  SPOs.  In  addition, 
the  AFTOC  plans  and  programs  division  could  help  by  ensuring 
that  the  recommendation  for  the  establishment  of  technical 
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orders  as  a  separate  product  is  properly  implemented.  This 
division  could  also  research  validation  and  verification 
problems  in  an  effort  to  identify  other  potential  solutions 
to  those  problems. 

The  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  provides  technical  order  validation  and 
verification  assistance  to  the  SPOs.*  Inputs  from  that 
organization  should  be  included  in  any  investigation  into 
this  problem. 

Obtaining  Technical  Orders  on  Time.  Technical  orders 
may  not  always  be  completed  when  the  equipment  that  they 
support  is  delivered.  When  this  happens,  it  is  usually  a 
result  of  the  planners  not  allowing  enough  time  to  develop, 
validate,  and  verify  technical  orders. 

The  actions  of  the  AFTOC  and  the  establishment  of 
technical  orders  as  a  separate  product  could  help  solve  this 
problem.  The  implementation  plans  for  the  AFTOC  and  for  the 
establishment  of  technical  orders  as  a  separate  product 
should  address  this  problem. 

Priority  of  Technical  Order  Acquisition.  When  a 
program  manager  considers  critical  factors  of  cost  and 
schedule  that  might  prevent  his  system  from  reaching 
completion,  technical  orders  are  not  generally  a  factor.  As 
a  result,  technical  order  acquisition  has  a  relatively  low 
priority  in  system  acquisition.  However,  system  acquisition 
managers  should  insure  that  technical  order  acquisition  is 
given  enough  priority  to  allow  for  the  timely  acquisition  of 
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accurate  and  complete  technical  orders.  Of  the  130 
respondents  interviewed  in  this  research,  14%  indicated  that 
the  relatively  low  priority  of  technical  order  acquisition 
within  the  system  acquisition  process  was  the  most 
significant  technical  order  acquisition  problem. 

The  establishment  of  technical  orders  as  a  separate 
product  could  help  raise  the  priority  of  technical  orders  by 
making  them  a  separate  contractual  obligation. 

The  AFTOC  could  also  help  increase  the  priority  of 
technical  orders  within  the  SPOs  through  the  increased 
technical  order  acquisition  assistance  that  it  could  provide 
to  SPO  personnel. 

The  implementation  plans  for  the  AFTOC  and  for  the 
establishment  of  technical  orders  as  a  separate  product 
should  address  this  problem. 

Chapter  Summary 

The  implementation  of  the  five  recommended  solutions 
identified  in  this  research  was  recommended.  An  integrated 
implementation  plan  for  those  recommended  solutions  was  also 
presented.  The  establishment  of  the  Air  Force  Technical 
Order  Center  was  the  key  to  that  recommended  plan.  The 
requirements  for  the  implementation  of  the  other  recommended 
solutions  will  be  affected  by  the  final  implementation  plan 
selected  for  the  AFTOC. 

Th**  AFTOC  should  have  responsibility  for  plans  and 
programs  for  the  technical  order  acquisition  process.  It 


should  include  technical  specialists#  and  it  should  provide 
field  assistance  to  the  SPOs. 

The  need  to  consider  technological  advances#  validation 
and  verification  problems#  timing  of  technical  order  de¬ 
livery#  and  technical  order  acquisition  priority  while 
developing  implementation  plans  for  the  five  recommended 
solutions  was  also  discussed. 

These  recommendations  are  offered  as  a  starting  point 
for  improving  the  technical  order  acquisition  process.  They 
should  be  considered  by  policy  makers  and  planners#  but 
should  not  be  implemented  without  further  research. 


Appendix  A:  Telephone  Interview  Schedule 


Introduction*  Good  morr ,  ng/aftemoon.  I  am  Capt  , 

a  graduate  student  at  the  AF  Institute  of  Technology.  I  am 
doing  research  concerning  the  TO  acquisition  process,  and 
you  can  help.  You  were  asked  to  participate  because  of  your 
position  as  [CTOM/PM/DPML/TOM] .  Your  responses  to  my 
questions  will  be  kept  anonymous,  arid  will  be  combined  with 
the  responses  of  others  in  positions  similar  to  yours.  The 
purpose  of  my  research  to  to  learn  more  about  the  TO 
acquisition  process,  and  your  candid  responses  will  help  me 
do  this.  Please  consider  only  the  acquisition  process  for 
new  systems  when  giving  your  responses.  Do  you  have  any 
questions  before  we  begin? 

1.  What  do  you  think  is  the  largest  single  problem  with  the 
TO  acquisition  process  as  it  is  today? 

2.  If  you  could  make  one  change  within  the  TO  acquisition 
process,  what  would  that  one  change  be? 

The  following  questions  are  related  specifically  to  the 
areas  of  interest  in  our  research.  Due  to  our  method  of 
gathering  statistics  for  the  following  questions,  we  need  a 
*  »yeg«  or  "no"  answer)  however,  please  feel  free  to  add  to  or 
embellish  any  answer  as  I  am  taking  notes  on  your  comments. 

3.  Do  you  think  planning  for  TOs  begins  early  enough  in  the 
acquisition  cycle? 

4.  Do  you  think  that  coordination  and  communication  between 
all  agencies  involved  in  the  TO  acquisition  process  for 
new  systems  is  adequate? 

5.  Do  you  think  the  manpower  dedicated  to  TO  acquisition  is 
adequate? 

6.  Do  you  think  the  training  and  assistance  that  TO 
acquisition  personnel  receive  is  adequate? 

7.  Do  you  think  the  establishment  of  a  centralized  AF  TO 
management  center  will  improve  the  TO  acquisition 
process? 

8.  Do  you  think  that  the  development  and  use  of  "skeleton" 
Statement  of  Work  (SOWs),  Requests  for  Proposals  (RFPs), 
and  contracts  will  significantly  help  TO  acquisition 
personnel? 
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.  Do  you  think  if  TOs  were  treated  as  a  "product"  to  be 
included  as  a  separate  line  item  in  the  SOW,  RFP,  and 
contract,  that  the  TO  acquisition  process  would  be 
improved? 

10.  Do  you  think  a  handbook  identifying  coordination  and 
cooperation  responsibilities  would  aid  TO  acquisition 
personnel? 

11.  Do  you  think  that  a  separate  specific  career  field  for 
TO  acquisition  managers  would  aid  the  TO  acquisition 
process? 

This  ends  the  structured  portion  of  the  interview.  Do 
you  have  anything  you  wish  to  add? 


EXTRA  NARRATIVE  FOR  INTERVIEWS  WITH  DPMLS: 

In  doing  this  research  we  were  provided  with  your  name, 
and  the  names  of  other  DPMLs .  To  complete  our  research,  we 
will  also  speak  to  Program  managers  and  TO  Mam  fers; 
however,  our  information  on  their  specific  names  and  phone 
numbers  is  not  exact.  Would  you  help  us  further  by 
providing  the  name  and  phone  number  of  your  PM,  and  the  same 
information  for  your  TOMAs,  or  those  personnel  working  for 
you  specifically  responsible  for  TO  acquisition?  Again, 
your  responses  to  our  questions  are  confidential  and  will 
not  be  discussed  with  other  respondents.  We  will  ask  them 
the  same  questions  we  have  just  asked  you. 

Thank  you  very  much  for  your  assistance.  Good  day! 
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Appendix  Bi  Question  One  Responses 


CTOM  Responses 

-  there  is  not  enough  understanding/training/education 

-  the  process  is  not  done  soon  enough 

-  there  is  no  centralized  OPR  (top  mgmt)  to  supervise 
decentralized  process 

-  TO  requirements  are  not  identified  to  contractors 

-  there  is  a  lack  of  senior  officer  mgmt  attention 

-  there  is  a  lack  of  control  by  acquiring  agencies 

-  there  is  no  centralized  downward  mgmt:  split  mgmt 

-  there  is  a  lack  of  education 

-  there  is  a  lack  of  staff  organization 

-  there  is  a  lack  of  TOMA  expertise  in  new  SPOs 

-  the  contract  structure  is  poor:  specs  bad 


PM  Responses 


ASP 

-  TOs  are  not  .getting  into  field  with  equipment 

-  validation/verification  is  late/bad 

-  focus  is  on  wrong  objectives:  is  process  instead  of  user 
oriented 

-  money  is  often  cut:  we  are  not  buying  right  stuff 

-  commercial  data  is  not  coupled  with  AF  needs 

-  ALC  knows  needs,  but  getting  'word'  to  contractor  is 
difficult 

-  time  required  to  get  TOFCNs  processed  is  excessive 

-  TOs  are  not  written  to  cover  subject:  milspecs  are  bad 

-  confusion  exists  because  different  services  buy  separate 
formats 

-  SPOs  are  not  manned  properly 

-  TOs  are  not  out  in  time 

-  this  is  a  creative  field,  and  we  don't  control  who  enters 
ESP 

-  timing:  TOs  get  out  too  late  for  maintenance  use 

-  documentation  requirements  are  cumbersome 

AD 

-  there  are  difficulties  between  organizations 
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-  the  hardware  changes  often,  causing  delays 


ASP 

-  TOs  do  not  get  into  field  for  ver.  before  maint.  uses  them 

-  there  is  a  lack  of  understanding  of  commercial  TOs 

-  the  TCTO  process  is  bad 

-  too  much  time  is  required  to  develop,  val/ver,  etc., 
especially  with  concurrency 

-  funding  is  low:  there  are  difficulties  with  scoping  the  TO 
down  ie:  use  commercial  or  milstd? 

-  it  is  difficult  to  find  qualified  TO  writers 

-  it  is  difficult  to  deal  with  the  magnitude  of  entire 
effort,  satisfying  all  requirements 

-  it  is  difficult  to  definite  specific,  vs  generalized 
requirements 

-  the  people  who  do  the  work  lack  experience 

-  achieving  timely  delivery  of  TOs  with  equipment  is 
difficult 

-  the  process  doesn't  get  as  much  emphasis  as  it  deserves 

-  it  is  difficult  to  insure -data  is  available  for  TOs  and  on 
time 

-  there  is  not  enough  serious  consideration  early  in  process 

-  verification  is  not  adequate 

-  PMRT  causes  problems 

-  acquiring  vendor  manuals  is  difficult 

-  there  is  a  lack  of  qualified  TOMAs 

-  there  is  no  identification  or  establishment  of  coord, 
between  gov.  &  contractor  (monolithic  mentality) 

-  it  is  difficult  to  'cost'  the  tech  data  S.  val/ver 
inadequate 

-  personnel  do  not  have  enough  experience 

-  val/ver  is  bad 

ESP 

-  the  ALC  is  unable  to  handle  commercial  products 

-  it  is  difficult  to  figure  out  costing  data 

-  manpower  limitations  prevent  development  of  functional 
experts 

-  the  contractor  does  not  know  AF  TO  acquisition  process 

-  people  are  not  properly  trained 

-  it  is  difficult  to  determine  how  much  they  should  do  & 
cost:  "inefficient  as  all  get-out" 

-  the  process  takes  too  long 

-  cost  is  high,  timing  with  equipment  in  the  field  is  poor 
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-  it  takes  too  long  to  create  a  TO  and  the  inability  to  use 
commercial  TOs 

-  there  is  no  solid  maint.  concept  at  the  outset #  and  LSA 
fails  to  drive  TO  development 

-  the  schedule  for  getting  hardware  S.  TOs  does  not  match 

-  timing  with  equipment  in  field  is  bad 

-  preplanning  the  cost  of  data  and  TOs  is  difficult 

-  hardware  design  changes  too  frequently 

-  the  time  frame  of  getting  drawings  from  contractor  is  bad 

-  there  are  difficulties  with  acquisition  of  commercial 
datat  changes  are  difficult 

-  the  timing  with  equipment  in  field  not  good 

-  the  review  by  using  and  supporting  commands  prior  to  use 
is  inadequate 


AD 

-  personnel  do  not  understand/  or  are  unable  to  evaluate# 
contractors  capabilities 

-  it  costs  too  much  money  for  what  we're  getting 

-  there  is  a  lack  of  definite  procedures  to  acquire  TOs 

•  the  process  is  too  slow:  then  is  not  initially  correct 

-  there  is  a  lack  of  trained  people 

-  because  LSA  &  LSAJR  are  not  fully  on  line  yet#  manning  is 
too  low  for  proper  support 

-  there  is  no  DOD  standard 

BMP 

-  it  is  difficult  to  determine  up  front  what  is  needed 

-  everything  written  today  refers  to  aircraft 

-  there  is  too  fine  a  line  between  starting  too  early  and 
not  early  enough 


TOMA  Responses 


ASP 

-  the  emphasis  is  on  equipment#  can  do  support  later 

-  it  is  diffi lit  to  get  TOs  verified  by  AF 

-  there  is  a  lack  of  good  direction 

-  getting  TOs  to  field  at  same  time  as  equipment  is 
difficult 

-  milspecs  are  outdated#  not  real-world 

-  there  is  no  continuity  in  TO  acq.  man.  career  field 

-  funds  are  inadequate 

-  the  funding  up  front  is  not  realistic 

-  there  is  a  lack  of  qualified  people 

-  personnel  are  unable  to  find  answers  to  questions 

-  management  structure  lacks  problem  solvers 
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-  TOMA  training  is  inadequate 

-  the  milspecs  are  outdated 

-  there  is  no  centralized  acq.  man.  body  to  help  SPOs 

-  there  is  no  unified  approach  by  AF 

-  TOs  are  not  considered  until  last  item 

-  policy  setters  have  no  TOMA  experience 

-  TOs  start  too  late  in  process 

-  TOs  are  non-standardized 

-  there  is  no  automated  tracking  system  available  to  TOMA 

-  TOMAs  come  into  ASD  without  'staff  experience 

-  there  is  a  lack  of  qualified  people  that  know  requirements 

-  the  method  by  which  we  release  data  (the  form)  is  poor 

-  how  we  do  business  is  not  standardized*  guidance  is  too 
generalized 

-  there  is  a  lack  of  money  for  support  equipment 

-  knowledge  of  data  requirements  by  those 
acquiring  TOs  is  inadequate 

-  TOs  do  not  get  verified  in  required  time 

ESP 

-  there  is  usually  no  maintenance  plan  up  front 

-  there  is  [confusion  with  milspecs  over  whether  all 
requirements  are  applied 

-  there  is  a  lack  of  manpower 

-  time  constraints  are  too  tight*  not  enough  available  for 
val/ver,  etc. 

-  there  is  [a  difference  of  opinion  between  AF  and  contractor 
as  to  what  is  a  valid  IPR 

-  it  is  difficult  to  get  contractor  to  comply*  no  leverage 
available  to  TO  manager 

-  not  enough  time  is  allowed  for  val/ver 

-  doing  process  on  both  Military  and  commercial  TOs  is 
difficult! 

-  there  is  |a  lack  of  coord,  between  various  organizations 
and  the  people  involved 

-  the  process  is  too  complex 

-  contractors  do  not  get  good  guidance  from  PM  office 

-  players  change  too  often:  same  people  don't  go  to  meetings 

-  difficult  to  maintain  currency  with  the  equipment  changes 

-  configuration  control  is  bad:  manual  doesn't  look  like  the 
equipment 

-  not  everyone  is  properly  trained 

-  there  is  a  lack  of  TDY  funds  by  ALC 

-  normal  process  is  okay:  problems  occur  when  TO  need  is 
accelerated 


AD 

-  there  is  no  continuity  of  expertise 

-  there  is  no  centralized  control  over  AF  TOs 

-  the  majority  of  PMs  &  logisticians  don’t  listen  to  TOMA 

-  milspecs  are  poorly  written  as  guidelines,  not  standards 
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-  there  are  too  many  governing  regs  and  rules 

-  TOMAs  have  little  authority  to  mako  decisions 

-  there  is  no  training  for  personnel 

BMP 

-  the  equipment  is  not  baselined  soon  enough 

-  it  is  difficult  to  keep  the  TO  matched  to  the  actual 
equipment 


Appendix  Cs  Question  Two  Responses 


CTOH  Responses 

-  ve  need  to  develop  automated/realtime  capabilities 

-  TOs  should  be  considered  earlier  in  the  process 

-  need  a  permanent/full  spectrum  centralized  mgmt 

•  should  fund  TOs  and  line  items  together,  but  as  separate 
line  items 

-  there  should  be  dedicated  manpower 

-  there  should  be  a  centralized  agency 

-  we  should  centralize  policy/mgmt 

-  there  should  be  less  centralization 

•  there  should  be  dedicated  manpower 

-  there  should  be  dedicated  manpower 

•  we  should  change  directives  S.  contract  requirements 


PM  Responses 


ASD 

•  do  something  to  get  TOs  to  field  same  time  as  equipment 

-  val/ver  should  be  improved 

-  we  should  cause  the  system  to  focus  on  user,  not  process 

-  increase  the  nucleus  of  knowledgeable  people 

-  improve  user  flexibility:  reliance  on  mil3pec  is  a  crutch 

-  simplify  the  process 

-  expedite  the  TOFCN  process 

-  let  TOs  be  developed  by  subject,  by  contractor 

-  have  better  manning:  use  technical  types  for  val/ver,  etc. 

-  require  more  training  for  TOMAs  and  get  more  semanticists 
and  learning  psychologists 

ESP 

-  improve  TO  timing  with  field  use  of  the  system 

-  make  sure  TO  development  is  closely  tied  to  system 
development 


AD 


-  form  a  centralized  agency  and  standardize 
BMP 

-  press  for  discipline  on  weapon  [engineering]  side 
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DPML  Responses 


ASP 

-  find  a  way  to  shorten  time  required  for  publication 
"try  to  maintain  continuity  in  a  closed-loop  system" 

-  form  a  centralized  agency 

-  computerize  the  TO  and  TCTO  process 

-  form  a  centralized  agency 

-  develop  TOs  earlier  in  the  process 

-  contract  for  the  acquisition  process  itself 

-  delay  writing  TOs  until  the  system  is  baselined 

-  simplify  coordination  between  agencies 

-  use  overall  engineering  data  bank  for  design  of  TOs 

-  get  more  knowledgeable  people  working  the  process 

-  identify  problems  early  enough  to  allow  fixes 

-  should  have  early  verification 

-  ensure  the  user  provides  equipment  for  val/ver 

-  clarify  early  what  constitutes  formal  transfer  from  ASP  to 
AFLC 

-  increase  the  number  of  TOMAs 

-  establish  common  requirements*  one  voice  from  government 

-  use  more  representative  people  in  val/ver 

-  keep  people  correctly  trained  so  they  can  do  the  job 
correctly 

-  improve  coordination  of  equipment  development  and  TO 
development 

ESP 

-  allow  for  development  of  commercial  manuals  on  a  separate 
track 

-  insist  on  use  of  system  that  is  AF  std,  not  company 
specific 

-  create  a  functional  specialist  on  TOs 

-  get  a  better  handle  on  number  of  TO  requirements 

-  automate  process 

-  automate  all  TOs*  allow  one  terminal  instead  of  TO  kit 

-  combine  LSA  with  TO  construction 

-  keep  the  books  in  preliminary  status  longer  to  allow 
corrections 

-  make  the  contractor  responsible  for  updates  S.  false  info 
damages 

-  delete  the  milspecs 

-  shorten  the  time  between  val/ver  and  final  distribution  to 
field 

-  improve  the  identification  of  TO  requirements 

-  get  logistics  more  involved  in  engineering 

-  move  toward  Army  'NEWLOOK'  (picture  intensive)  system 

-  use  commercial  off-the-shelf  equipment  data 
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-  combine  validation  and  verification 

-  maintain  user-type  experience  in  offices  for  review 
process 

AD 

-  find  better  qualified  sources 

-  identify  where  hard  copies  cure  necessary 

-  should  use  trained/  knowledgeable  people  to  deal  with 
contractors 

-  automate  entire  process  so  changes  can  be  real-time 

-  improve  process  of  getting  TOs  published  (material  mgmt) 

-  the  process  is  working  real  well! 

-  develop  a  DOD  standard 

BMP 

-  write  stricter  contracts 

-  automate  everything:  change  specs 

-  use  knowledgeable  people 


TOMA  Responses 


ASP 

-  make  contractor  supply  TOs  with  equipment  or  no  money 

-  consolidate  'how-to'  info  in  one  book 

-  have  verification  earlier 

-  automate  delivery  of  TOs 

-  provide  continuity  of  personnel 

-  elevate  TO  function  to  that  of  others  (2-letter  office) 

-  have  a  larger  interchange  with  industry  methods 

-  develop  one  standard  document  for  total  process  that  can 
be  tailored 

-  form  a  centralized  agency 

-  form  a  centralized  agency 

-  update  milspecs  to  the  level  of  today's  technology 

-  form  a  centralized  agency,  plus  improve  training 

-  develop  and  identify  a  standard  approach 

-  see  that  TOs  are  included  up  frontlin  the  process 

-  make  TOs  separate  division  under  PM  to  increase 

recognition  j 

-  provide  better  guidelines  on  participants  functions 

-  should  be  more  enforcement  of  contract  to  reduce  late 

deliveries  \ 

-  find  a  way  to  attract  qualified  people  to  be  TOMAs:  CBPO 
could  flag  possible  candidates 

-  make  sure  TOMAs  have  school,  training,  etc. 

-  TO  manager  should  have  field  TO  expedience 

-  institute  a  way  to  electrically  transmit  data  to  user 
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-  make  TO  part  of  LSA  automated,  on-line  system  at  SPO, 
cont. ,  ALC,  &  MAJCOM 

-  add  emphasis *  all  TOs  for  a  system  should  be  available 
with  mission  equipment 

-  streamline  printing/distribution/changing  procedures 
ESP 

-  have  a  maintenance  plan  early 

-  use  of  AFADS  should  be  clarified,  then  try  process  again 

-  provide  more  manpower  on  job 

-  allocate  more  manpower  to  Tech  Data  organizations 

-  have  a  more  detailed  verification  effort 

-  find  a  way  to  make  contractor  comply 

-  change  printing  requirements  from  negatives  to  std  8.5x11 

-  keep  'TO  team'  together  vice  different  people  each  meeting 

-  find  a  way  to  get  contractor  to  agree  to  prices 

-  clean  up  CDRLs  &  AFADs:  perhaps  simplify 

-  introduce  pre-guidance ,  have  less  structure 

-  pare  down  or  consolidate  mil  specs 

-  simplify  the  process 

-  monitor  process  more  closely:  do  better  job  of 
verification 

-  update  milspecs:  paying  for  much  unneeded  info 

-  make  reviews  mandatory  by  ALC,  and  include  safety  reps 


-  develop  a  system  to  track  people  with  TO  acq.  experience 

-  develop  a  responsible  local  and  staff  management 

-  should  buy  manuals,  not  TOs 

-  form  a  centralized  agency  under  Air  Staff 

-  consolidate  the  guidelines 

-  increase  training  for  personnel 

-  delegate  more  decision  authority  to  TOMAs 

BMP 

-  streamline  the  ordering  process:  perhaps  use  computers 

-  improve  interface  between  test  people  and  TO  people 
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Bachelor  of  Arts  in  General  Studies  from  Eastern  Illinois 
University.  He  was  selected  to  attend  Officer  Training 
School/  from  which  he  was  a  Distinguished  Graduate  in  August 
1979.  He  was  trained  as  a  missile  maintenance  officer,  and 
last  served  at  Whiteman  AFB/  Missouri,  as  the  Officer  in 
Charge  of  the  Job  Control  Branch,  until  entering  the  School 
of  Systems  and  Logistics,  Air  Force  Institute  of  Technology, 
in  May  1983. 
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The  objective  of  this  rsseareh  was  to  identify  problaas 
with  tha  acquisition  of  U.  S.  Air  Force  technical  orders 
(70s),  and  to  Identify  changes  to  the  TO  acquisition  process 
that  could  solve  those  problems. 

The  objective  was  accomplished  through  a  telephone  sur¬ 
vey  of  AF  policy  makers.  Program  Managers,  Integrated  Logis¬ 
tics  Support  managers,  and  TO  acquisition  managers. 

The  results  of  the  telephone  survey  indicate  that  the 
low  experience  level  of  personnel  assigned  to  TO  acquisition 
responsibilities,  and  coordinatlon/eommunicatlon  problems 
were  the  most  significant  problems  with  the  TO  acquisition 
process.  Inadequate  manning  and  the  lack  of  early  planning 
for  TO  acquisition  were  also  found  to  be  problems. 

Five  identified  solutions  to  those  problems  were  found 
to  be  valid  and  were  recommended  for  implementation.  They 
are  the  establishment  of  a  centralised  TO  management  agency, 
treatment  of  70s  as  a  separate  product  during  weapon  system 
acquisition,  the  use  of  "skeleton1*  documents,  the  develop¬ 
ment  of  a  handbook  outlining  responsibilities,  and  the  estab¬ 
lishment  of  a  separate  career  field  for  TO  acquisition  man¬ 
agers.  The  establishment  o<  a  centralized  TO  management 
agency  was  found  to  be  the  most  needed  solution.  A  recom¬ 
mended  Implementation  plan  was  presented  for  each  solution. 
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